Add batch plans for batches 31-36 (rounds 16-18)
Generated design docs and implementation plans via Codex for: - Batch 31: Raft Part 2 - Batch 32: JS Cluster Meta - Batch 33: JS Cluster Streams - Batch 34: JS Cluster Consumers - Batch 35: JS Cluster Remaining - Batch 36: Stream Lifecycle All plans include mandatory verification protocol and anti-stub guardrails. Updated batches.md with file paths and planned status.
This commit is contained in:
Joseph Doherty
156
docs/plans/2026-02-27-batch-36-stream-lifecycle-design.md
Normal file
156
docs/plans/2026-02-27-batch-36-stream-lifecycle-design.md
Normal file
@@ -0,0 +1,156 @@
|
||||
# Batch 36 Stream Lifecycle Design
|
||||
|
||||
**Date:** 2026-02-27
|
||||
**Batch:** 36 (`Stream Lifecycle`)
|
||||
**Scope:** 92 features + 53 unit tests
|
||||
**Dependencies:** Batches `8, 11, 12, 13, 14, 15, 28`
|
||||
**Go source:** `golang/nats-server/server/stream.go` (focus through ~line 4600)
|
||||
|
||||
## Problem
|
||||
|
||||
Batch 36 is the lifecycle/core control plane for JetStream streams: stream creation, assignment, config validation/update, purge/delete flows, mirror/source setup and message processing, and internal subscriptions. This batch is a gating dependency for Batch 37 (`Stream Messages`) and Batch 38 (`Consumer Lifecycle`), so fake progress here creates downstream breakage.
|
||||
|
||||
The current .NET surface is materially under-ported for this area:
|
||||
|
||||
- Batch 36 features are all `deferred`.
|
||||
- Most mapped methods are missing from source classes.
|
||||
- Existing `ImplBacklog` tests for related classes are largely placeholder-style and need replacement with behavior checks.
|
||||
|
||||
## Context Findings
|
||||
|
||||
### Required command outputs
|
||||
|
||||
- `/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- batch show 36 --db porting.db`
|
||||
- Status: `pending`
|
||||
- Features: `92` (`3196-3293` with gaps)
|
||||
- Tests: `53`
|
||||
- Depends on: `8,11,12,13,14,15,28`
|
||||
- Go file: `server/stream.go`
|
||||
- `/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- batch list --db porting.db`
|
||||
- Confirms chain: `36 -> 37,38`
|
||||
- `/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- report summary --db porting.db`
|
||||
- Overall progress: `1924/6942 (27.7%)`
|
||||
|
||||
Environment note: `dotnet` is not on `PATH` in this shell. Use `/usr/local/share/dotnet/dotnet` explicitly.
|
||||
|
||||
### Feature ownership split (from `porting.db`)
|
||||
|
||||
- `NatsStream`: 76
|
||||
- `Account`: 4
|
||||
- `PersistModeType`: 3
|
||||
- `StreamSourceInfo`: 2
|
||||
- `StreamSource`: 2
|
||||
- `JsAccount`: 2
|
||||
- `NatsServer`: 1
|
||||
- `InMsg`: 1
|
||||
- `ExternalStream`: 1
|
||||
|
||||
### Test split (from `porting.db`)
|
||||
|
||||
- `JetStreamEngineTests`: 35
|
||||
- `NatsConsumerTests`: 4
|
||||
- `JetStreamClusterTests1`: 3
|
||||
- `JetStreamClusterTests2`: 4
|
||||
- `JetStreamClusterTests3`: 3
|
||||
- `ConcurrencyTests1`: 2
|
||||
- `JetStreamFileStoreTests`: 1
|
||||
- `StorageEngineTests`: 1
|
||||
|
||||
Additional finding: `JetStreamClusterTests1.Impltests.cs` and `JetStreamClusterTests3.Impltests.cs` do not currently exist and must be created.
|
||||
|
||||
## Approaches
|
||||
|
||||
### Approach A: Single-file port in existing files only
|
||||
|
||||
Port all methods directly into existing files (`NatsStream.cs`, `Account.cs`, `JetStreamTypes.cs`, etc.) without structural splits.
|
||||
|
||||
- Pros: minimal file churn.
|
||||
- Cons: high merge risk, hard reviewability, and poor fault isolation for 76 `NatsStream` methods.
|
||||
|
||||
### Approach B (Recommended): Lifecycle-focused partial-file decomposition + staged verification
|
||||
|
||||
Keep mapped owning classes intact but split implementation by concern into new partial files for stream lifecycle/mirror/source/subscription paths. Execute in bounded feature groups and test waves with strict evidence gates.
|
||||
|
||||
- Pros: manageable review units, clearer ownership, better checkpointing and rollback, aligns with anti-stub guardrails.
|
||||
- Cons: requires adding partial declarations where classes are currently non-partial.
|
||||
|
||||
### Approach C: Test-first broad wave before feature completion
|
||||
|
||||
Attempt to port all 53 tests up front to drive implementation.
|
||||
|
||||
- Pros: fast behavior pressure.
|
||||
- Cons: most tests depend on missing lifecycle internals and will churn heavily; expensive red-state noise.
|
||||
|
||||
**Decision:** Approach B.
|
||||
|
||||
## Proposed Design
|
||||
|
||||
### 1. Code organization strategy
|
||||
|
||||
Use concern-oriented partial files while preserving mapped class ownership:
|
||||
|
||||
- `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.cs` (retain core type; convert to `partial`)
|
||||
- Create `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.Lifecycle.cs`
|
||||
- Create `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.Mirror.cs`
|
||||
- Create `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.Source.cs`
|
||||
- Create `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.Subscriptions.cs`
|
||||
- `dotnet/src/ZB.MOM.NatsNet.Server/Accounts/Account.cs` (convert to `partial`)
|
||||
- Create `dotnet/src/ZB.MOM.NatsNet.Server/Accounts/Account.StreamLifecycle.cs`
|
||||
- `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamTypes.cs` (for `JsAccount`, convert class to `partial` if split)
|
||||
- Create `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JsAccount.StreamLifecycle.cs`
|
||||
- `dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.cs` or `NatsServer.*.cs` partial for `CheckStreamCfg`
|
||||
- `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/StoreTypes.cs` and `StreamTypes.cs` for `PersistModeType`, `ExternalStream.Domain`, `StreamSource`/`StreamSourceInfo`, `InMsg`
|
||||
|
||||
### 2. Functional decomposition
|
||||
|
||||
- **Config and creation path:** `PersistModeType`, `ExternalStream.Domain`, `Account.AddStream*`, `StreamSource.ComposeIName/SetIndexName`, `NatsServer.CheckStreamCfg`, `JsAccount.ConfigUpdateCheck`, `NatsStream.Update*`.
|
||||
- **State/advisory path:** leader checks, sequence counters (`CLFS`), created time, create/update/delete advisories, purge/delete/erase behavior.
|
||||
- **Mirror path:** mirror setup/retry/cancel, inbound mirror flow-control handling, lag tracking, retry backoff.
|
||||
- **Source path:** source setup/retry/cancel, shared source message pump, source headers, start-sequence reconstruction.
|
||||
- **Subscription/internal path:** stream/direct/mirror-direct subscriptions, source consumer stop/unsubscribe, batching cleanup, internal subscribe helpers.
|
||||
|
||||
### 3. Test design
|
||||
|
||||
Port test IDs in waves tied to implemented behavior:
|
||||
|
||||
- Wave 1: file store + cluster/meta + consumer API basics (15 tests)
|
||||
- Wave 2-4: `JetStreamEngineTests` heavy stream lifecycle scenarios (12/12/14)
|
||||
|
||||
Test files:
|
||||
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamEngineTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/NatsConsumerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests2.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamFileStoreTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests1.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/StorageEngineTests.Impltests.cs`
|
||||
- Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests1.Impltests.cs`
|
||||
- Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests3.Impltests.cs`
|
||||
|
||||
### 4. Verification and risk controls (design-level)
|
||||
|
||||
- Per-feature and per-test loops are mandatory before status promotion.
|
||||
- Stub scans are mandatory after each loop and at each checkpoint.
|
||||
- Build and test gates are mandatory before every status batch update.
|
||||
- Status updates are chunked to max 15 IDs.
|
||||
- If blocked: remain `deferred` with explicit reason, never placeholder implementation.
|
||||
|
||||
## Feature Grouping (for implementation plan)
|
||||
|
||||
- Group A (20): IDs `3196-3219` (actual mapped IDs, 20 total)
|
||||
- Group B (20): IDs `3220-3240` (actual mapped IDs, 20 total)
|
||||
- Group C (20): IDs `3241-3261` (actual mapped IDs, 20 total)
|
||||
- Group D (20): IDs `3262-3281` (actual mapped IDs, 20 total)
|
||||
- Group E (12): IDs `3282-3293` (actual mapped IDs, 12 total)
|
||||
|
||||
## Constraints
|
||||
|
||||
- Planning only in this session; no implementation execution.
|
||||
- Must preserve .NET standards (`xUnit 3`, `Shouldly`, `NSubstitute`, nullable, naming conventions).
|
||||
- Must avoid fake-pass tests and placeholder feature bodies.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Executing Batch 36 implementation in this session.
|
||||
- Expanding scope beyond Batch 36 mapped features/tests.
|
||||
- Building new integration infrastructure not required by mapped unit tests.
|
||||
Reference in New Issue
Block a user