Files
natsnet/docs/plans/2026-02-27-batch-28-jetstream-api-implementation-plan.md
Joseph Doherty c05d93618e Add batch plans for batches 23-30 (rounds 12-15)
Generated design docs and implementation plans via Codex for:
- Batch 23: Routes
- Batch 24: Leaf Nodes
- Batch 25: Gateways
- Batch 26: WebSocket
- Batch 27: JetStream Core
- Batch 28: JetStream API
- Batch 29: JetStream Batching
- Batch 30: Raft Part 1

All plans include mandatory verification protocol and anti-stub guardrails.
Updated batches.md with file paths and planned status.
2026-02-27 16:33:10 -05:00

551 lines
20 KiB
Markdown

# Batch 28 JetStream API Implementation Plan
> **For Codex:** REQUIRED SUB-SKILL: Use `executeplan` to implement this plan task-by-task.
**Goal:** Port and verify Batch 28 JetStream API behavior (55 features, 2 tests) from `server/jetstream_api.go` with strict no-stub execution and evidence-based PortTracker status updates.
**Architecture:** Implement Batch 28 in three feature waves aligned to API lifecycle (dispatch/helpers, stream-control handlers, message/consumer/snapshot handlers), then port and verify the two mapped tests. Keep class/method mapping audit-safe (`NatsServer`, `JetStreamApi`, `JetStreamEngine`, `Account`, `JsAccount`) and enforce mandatory verification checkpoints before every status promotion.
**Tech Stack:** .NET 10, C# latest, xUnit 3, Shouldly, NSubstitute, PortTracker CLI, SQLite (`porting.db`)
**Design doc:** `docs/plans/2026-02-27-batch-28-jetstream-api-design.md`
---
## Batch 28 Scope
- Batch ID: `28`
- Name: `JetStream API`
- Dependencies: `5`, `27`
- Go source: `golang/nats-server/server/jetstream_api.go`
- Feature IDs: `1452,1454-1507` (55 total; `1453` not present)
- Test IDs: `1716`, `1719`
Primary implementation files:
- Create/Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamApi.cs`
- Create/Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamEngine.cs`
- Create/Modify: `dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.JetStreamApi.cs`
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.cs` (state fields only if required)
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/Accounts/Account.cs`
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamTypes.cs` (`JsAccount` methods)
Primary test files:
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamEngineTests.Impltests.cs`
- Create/Modify (supporting focused tests if needed): `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/JetStreamApiTests.cs`
---
## MANDATORY VERIFICATION PROTOCOL
> **NON-NEGOTIABLE:** Every feature and test in this batch must satisfy this protocol. Skipping steps is a plan violation.
### Preflight Dependency Gate (REQUIRED before any feature work)
1. Verify dependency batches are complete enough to start Batch 28:
```bash
dotnet run --project tools/NatsNet.PortTracker -- batch show 5 --db porting.db
dotnet run --project tools/NatsNet.PortTracker -- batch show 27 --db porting.db
dotnet run --project tools/NatsNet.PortTracker -- batch ready --db porting.db
```
2. Start batch only when ready:
```bash
dotnet run --project tools/NatsNet.PortTracker -- batch start 28 --db porting.db
```
3. Baseline gates:
```bash
dotnet build dotnet/
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
```
### Per-Feature Verification Loop (REQUIRED for each feature ID)
1. Read mapped feature details and Go span:
```bash
dotnet run --project tools/NatsNet.PortTracker -- feature show <feature_id> --db porting.db
sed -n '<go_line_start>,<go_line_end>p' golang/nats-server/server/jetstream_api.go
```
2. Mark the single active feature `stub` before coding:
```bash
dotnet run --project tools/NatsNet.PortTracker -- feature update <feature_id> --status stub --db porting.db
```
3. Implement only that feature behavior in mapped class/method.
4. **Build gate (per feature):**
```bash
dotnet build dotnet/
```
5. **Test gate (per feature):** run the smallest focused tests that execute the changed path:
```bash
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~JetStreamApi"
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~ImplBacklog.JetStreamEngineTests"
```
6. If both gates are green and no stub markers are introduced, promote feature to `complete`:
```bash
dotnet run --project tools/NatsNet.PortTracker -- feature update <feature_id> --status complete --db porting.db
```
7. Promote to `verified` only after group-level gates pass.
### Per-Test Verification Loop (REQUIRED for each test ID)
1. Read mapping and Go test body:
```bash
dotnet run --project tools/NatsNet.PortTracker -- test show <test_id> --db porting.db
sed -n '<go_line_start>,<go_line_end>p' golang/nats-server/server/jetstream_test.go
```
2. Port/replace one C# test method with real Arrange/Act/Assert behavior that calls production code.
3. Run the single test and confirm it is discovered:
```bash
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
--filter "FullyQualifiedName~ZB.MOM.NatsNet.Server.Tests.ImplBacklog.JetStreamEngineTests.<MethodName>" \
--verbosity normal
```
4. Validate summary contains `Passed: 1, Failed: 0` (not `Passed: 0`).
5. Only then set test status to `complete`; move to `verified` after class-level and batch-level gates.
### Stub Detection Check (REQUIRED after every feature group and after test edits)
Run against touched files only:
```bash
git diff --name-only -- dotnet/src/ZB.MOM.NatsNet.Server dotnet/tests/ZB.MOM.NatsNet.Server.Tests \
| rg "\.cs$" \
| xargs rg -n "(NotImplementedException|// TODO|// PLACEHOLDER|Assert\.True\(true\)|Assert\.Pass|throw new Exception\(\"TODO\"\))"
```
If any match appears in newly touched code, stop and fix or defer; do not promote status.
### Build Gate (REQUIRED)
Must pass:
- after every feature ID loop
- after every feature group
- before any `batch-update` status command
- after test task completion
Command:
```bash
dotnet build dotnet/
```
### Test Gate (REQUIRED)
Minimum required gates for Batch 28:
```bash
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~ImplBacklog.JetStreamEngineTests.IsJSONObjectOrArray_ShouldSucceed"
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~ImplBacklog.JetStreamEngineTests.JetStreamDelayedAPIResponses_ShouldSucceed"
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~ImplBacklog.JetStreamEngineTests"
```
At task checkpoints and final closure, run full unit test suite:
```bash
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
```
### Status Update Protocol (HARD LIMIT: max 15 IDs per batch-update)
- Allowed progress path: `deferred/not_started -> stub -> complete -> verified`
- Never update more than 15 feature IDs in one `feature batch-update` call
- Never update more than 15 test IDs in one `test batch-update` call
- Never set `verified` without captured build + test + stub-scan evidence
- If audit mismatch occurs, use explicit override reason only when evidence is documented
Templates:
```bash
dotnet run --project tools/NatsNet.PortTracker -- \
feature batch-update --ids "<max 15 ids>" --set-status <stub|complete|verified> --db porting.db --execute
dotnet run --project tools/NatsNet.PortTracker -- \
test batch-update --ids "<max 15 ids>" --set-status <stub|complete|verified> --db porting.db --execute
```
### Checkpoint Protocol Between Tasks (REQUIRED)
At the end of each task before moving on:
1. Run stub detection check.
2. Run build gate.
3. Run required test gate for touched classes.
4. Record updated IDs and evidence output.
5. Commit checkpoint.
---
## ANTI-STUB GUARDRAILS (NON-NEGOTIABLE)
### Forbidden Patterns (Features + Tests)
Any of the following in newly touched methods means the item is a stub and must not be promoted:
- `throw new NotImplementedException()`
- Empty mapped method body (`{ }`)
- Placeholder comments (`// TODO`, `// PLACEHOLDER`, `// later`)
- Fake success tests (`Assert.True(true)`, `Assert.Pass()`)
- Tests that do not call production code
- Trivial constant-return placeholders for non-trivial handlers (`return null;`, `return false;`, `return 0;`, `return string.Empty;`) without behavior-equivalent Go logic
### Hard Limits
- Max feature group size: `~20`
- Max IDs per `batch-update`: `15`
- One active feature implementation loop at a time
- No status promotion if build gate is red
- No status promotion if stub scan has unresolved matches
- No cross-group status updates (only IDs in current task)
### If You Get Stuck (MANDATORY)
1. Stop implementing the blocked ID immediately.
2. Do **not** leave placeholder code or fake tests.
3. Mark blocked item `deferred` with an explicit reason:
```bash
dotnet run --project tools/NatsNet.PortTracker -- feature update <id> --status deferred --override "blocked: <specific reason>" --db porting.db
dotnet run --project tools/NatsNet.PortTracker -- test update <id> --status deferred --override "blocked: <specific reason>" --db porting.db
```
4. Move to the next unblocked ID in the same task.
5. Deferred with reason is correct behavior; stubbing is not.
---
## Feature Group Map (Max ~20 per task)
### Group A (20 features): Dispatch, request parsing, account tracking, delayed responder core
`1452,1454,1455,1456,1457,1458,1459,1460,1461,1462,1463,1464,1465,1466,1467,1468,1469,1470,1471,1472`
### Group B (18 features): Stream CRUD/list/info and control-plane handlers
`1473,1474,1475,1476,1477,1478,1479,1480,1481,1482,1483,1484,1485,1486,1487,1488,1489,1490`
### Group C (17 features): Message/consumer APIs, purge/restore/snapshot, API advisory
`1491,1492,1493,1494,1495,1496,1497,1498,1499,1500,1501,1502,1503,1504,1505,1506,1507`
### Batch 28 Tests
`1716,1719`
---
### Task 1: Dependency Gate and Batch Initialization
**Files:**
- Read: `docs/standards/dotnet-standards.md`
- Read: `docs/plans/2026-02-27-batch-28-jetstream-api-design.md`
- Read: `golang/nats-server/server/jetstream_api.go`
**Step 1: Verify Batch 28 dependencies are complete**
Run:
```bash
dotnet run --project tools/NatsNet.PortTracker -- batch show 5 --db porting.db
dotnet run --project tools/NatsNet.PortTracker -- batch show 27 --db porting.db
dotnet run --project tools/NatsNet.PortTracker -- batch ready --db porting.db
```
Expected:
- Batch 28 appears ready (or dependencies explicitly complete enough to start).
**Step 2: Start Batch 28**
Run:
```bash
dotnet run --project tools/NatsNet.PortTracker -- batch start 28 --db porting.db
```
Expected:
- Batch status transitions to in-progress without dependency validation failure.
**Step 3: Baseline gates**
Run:
```bash
dotnet build dotnet/
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
```
Expected:
- Build succeeds; baseline failures (if any) are known before feature edits.
**Step 4: Commit checkpoint**
```bash
git add porting.db
git commit -m "chore(batch28): start jetstream api batch"
```
---
### Task 2: Implement Feature Group A (20 IDs)
**Files:**
- Create/Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamApi.cs`
- Create/Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamEngine.cs`
- Create/Modify: `dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.JetStreamApi.cs`
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/Accounts/Account.cs`
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamTypes.cs`
- Optional state wiring: `dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.cs`
**Step 1: Mark Group A as `stub` in chunks (max 15 IDs)**
```bash
dotnet run --project tools/NatsNet.PortTracker -- feature batch-update --ids "1452,1454,1455,1456,1457,1458,1459,1460,1461,1462,1463,1464,1465,1466,1467" --set-status stub --db porting.db --execute
dotnet run --project tools/NatsNet.PortTracker -- feature batch-update --ids "1468,1469,1470,1471,1472" --set-status stub --db porting.db --execute
```
**Step 2: Execute the Per-Feature Verification Loop for each Group A ID**
Expected:
- Each ID reaches `complete` individually only after build+test+stub-scan evidence.
**Step 3: Run Group A gates**
```bash
dotnet build dotnet/
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~ImplBacklog.JetStreamEngineTests"
```
Expected:
- Build succeeds and no regression in JetStream engine backlog class.
**Step 4: Promote Group A IDs to `complete` (chunked)**
```bash
dotnet run --project tools/NatsNet.PortTracker -- feature batch-update --ids "1452,1454,1455,1456,1457,1458,1459,1460,1461,1462,1463,1464,1465,1466,1467" --set-status complete --db porting.db --execute
dotnet run --project tools/NatsNet.PortTracker -- feature batch-update --ids "1468,1469,1470,1471,1472" --set-status complete --db porting.db --execute
```
**Step 5: Checkpoint protocol and commit**
```bash
git add dotnet/src/ZB.MOM.NatsNet.Server porting.db
git commit -m "feat(batch28): implement jetstream api dispatch and account request core"
```
---
### Task 3: Implement Feature Group B (18 IDs)
**Files:**
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.JetStreamApi.cs`
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamApi.cs`
- Modify (if needed): `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamTypes.cs`
**Step 1: Mark Group B as `stub` (chunked)**
```bash
dotnet run --project tools/NatsNet.PortTracker -- feature batch-update --ids "1473,1474,1475,1476,1477,1478,1479,1480,1481,1482,1483,1484,1485,1486,1487" --set-status stub --db porting.db --execute
dotnet run --project tools/NatsNet.PortTracker -- feature batch-update --ids "1488,1489,1490" --set-status stub --db porting.db --execute
```
**Step 2: Execute the Per-Feature Verification Loop for each Group B ID**
Expected:
- Stream and control handlers are fully behavior-ported, not placeholders.
**Step 3: Run Group B gates**
```bash
dotnet build dotnet/
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~ImplBacklog.JetStreamEngineTests"
```
**Step 4: Promote Group B IDs to `complete` (chunked)**
```bash
dotnet run --project tools/NatsNet.PortTracker -- feature batch-update --ids "1473,1474,1475,1476,1477,1478,1479,1480,1481,1482,1483,1484,1485,1486,1487" --set-status complete --db porting.db --execute
dotnet run --project tools/NatsNet.PortTracker -- feature batch-update --ids "1488,1489,1490" --set-status complete --db porting.db --execute
```
**Step 5: Checkpoint protocol and commit**
```bash
git add dotnet/src/ZB.MOM.NatsNet.Server porting.db
git commit -m "feat(batch28): implement jetstream api stream and control handlers"
```
---
### Task 4: Implement Feature Group C (17 IDs)
**Files:**
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.JetStreamApi.cs`
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/Accounts/Account.cs`
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamTypes.cs`
**Step 1: Mark Group C as `stub` (chunked)**
```bash
dotnet run --project tools/NatsNet.PortTracker -- feature batch-update --ids "1491,1492,1493,1494,1495,1496,1497,1498,1499,1500,1501,1502,1503,1504,1505" --set-status stub --db porting.db --execute
dotnet run --project tools/NatsNet.PortTracker -- feature batch-update --ids "1506,1507" --set-status stub --db porting.db --execute
```
**Step 2: Execute the Per-Feature Verification Loop for each Group C ID**
Expected:
- Consumer/message/snapshot and advisory handlers behave per Go source intent.
**Step 3: Run Group C gates**
```bash
dotnet build dotnet/
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~ImplBacklog.JetStreamEngineTests"
```
**Step 4: Promote Group C IDs to `complete` (chunked)**
```bash
dotnet run --project tools/NatsNet.PortTracker -- feature batch-update --ids "1491,1492,1493,1494,1495,1496,1497,1498,1499,1500,1501,1502,1503,1504,1505" --set-status complete --db porting.db --execute
dotnet run --project tools/NatsNet.PortTracker -- feature batch-update --ids "1506,1507" --set-status complete --db porting.db --execute
```
**Step 5: Checkpoint protocol and commit**
```bash
git add dotnet/src/ZB.MOM.NatsNet.Server porting.db
git commit -m "feat(batch28): implement jetstream api consumer/message/snapshot handlers"
```
---
### Task 5: Port and Verify Batch 28 Tests (2 IDs)
**Files:**
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamEngineTests.Impltests.cs`
- Optional support tests: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/JetStreamApiTests.cs`
**Step 1: Mark both tests `stub`**
```bash
dotnet run --project tools/NatsNet.PortTracker -- test batch-update --ids "1716,1719" --set-status stub --db porting.db --execute
```
**Step 2: Implement `IsJSONObjectOrArray_ShouldSucceed` (ID 1716) with table-style cases from Go**
Run:
```bash
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~IsJSONObjectOrArray_ShouldSucceed" --verbosity normal
```
Expected:
- `Passed: 1, Failed: 0`
**Step 3: Implement `JetStreamDelayedAPIResponses_ShouldSucceed` (ID 1719)**
Run:
```bash
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~JetStreamDelayedAPIResponses_ShouldSucceed" --verbosity normal
```
Expected:
- `Passed: 1, Failed: 0`
**Step 4: Class-level gate for `JetStreamEngineTests`**
```bash
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~ImplBacklog.JetStreamEngineTests" --verbosity normal
```
**Step 5: Promote tests to `complete`, then `verified`**
```bash
dotnet run --project tools/NatsNet.PortTracker -- test batch-update --ids "1716,1719" --set-status complete --db porting.db --execute
dotnet run --project tools/NatsNet.PortTracker -- test batch-update --ids "1716,1719" --set-status verified --db porting.db --execute
```
**Step 6: Checkpoint protocol and commit**
```bash
git add dotnet/tests/ZB.MOM.NatsNet.Server.Tests porting.db
git commit -m "test(batch28): port jetstream api object-shape and delayed-response tests"
```
---
### Task 6: Final Verification, Status Closure, and Batch Completion
**Files:**
- Modify: `porting.db`
- Generate: `reports/current.md` (via report script)
**Step 1: Full gates**
```bash
dotnet build dotnet/
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
```
**Step 2: Final stub scan on touched files**
```bash
git diff --name-only -- dotnet/src/ZB.MOM.NatsNet.Server dotnet/tests/ZB.MOM.NatsNet.Server.Tests \
| rg "\.cs$" \
| xargs rg -n "(NotImplementedException|// TODO|// PLACEHOLDER|Assert\.True\(true\)|Assert\.Pass)"
```
Expected:
- No unresolved matches in touched files.
**Step 3: Promote feature IDs to `verified` in max-15 chunks**
```bash
dotnet run --project tools/NatsNet.PortTracker -- feature batch-update --ids "1452,1454,1455,1456,1457,1458,1459,1460,1461,1462,1463,1464,1465,1466,1467" --set-status verified --db porting.db --execute
dotnet run --project tools/NatsNet.PortTracker -- feature batch-update --ids "1468,1469,1470,1471,1472,1473,1474,1475,1476,1477,1478,1479,1480,1481,1482" --set-status verified --db porting.db --execute
dotnet run --project tools/NatsNet.PortTracker -- feature batch-update --ids "1483,1484,1485,1486,1487,1488,1489,1490,1491,1492,1493,1494,1495,1496,1497" --set-status verified --db porting.db --execute
dotnet run --project tools/NatsNet.PortTracker -- feature batch-update --ids "1498,1499,1500,1501,1502,1503,1504,1505,1506,1507" --set-status verified --db porting.db --execute
```
**Step 4: Validate batch/report visibility**
```bash
dotnet run --project tools/NatsNet.PortTracker -- batch show 28 --db porting.db
dotnet run --project tools/NatsNet.PortTracker -- report summary --db porting.db
```
**Step 5: Complete batch and generate report**
```bash
dotnet run --project tools/NatsNet.PortTracker -- batch complete 28 --db porting.db
./reports/generate-report.sh
```
**Step 6: Final commit**
```bash
git add dotnet/src/ZB.MOM.NatsNet.Server dotnet/tests/ZB.MOM.NatsNet.Server.Tests porting.db reports/current.md
git commit -m "feat(batch28): complete jetstream api feature and test port"
```
---
## Execution Notes
- This plan is intentionally strict on status evidence because Batch 28 is a high-fanout dependency for later stream/consumer lifecycle batches.
- If dependency batch 27 remains incomplete when execution starts, Task 1 must stop early and keep Batch 28 unstarted; do not bypass dependency enforcement.