natsnet/docs/plans/2026-02-27-batch-36-stream-lifecycle-design.md

Batch 36 Stream Lifecycle Design

Date: 2026-02-27
Batch: 36 (Stream Lifecycle)
Scope: 92 features + 53 unit tests
Dependencies: Batches 8, 11, 12, 13, 14, 15, 28
Go source: golang/nats-server/server/stream.go (focus through ~line 4600)

Problem

Batch 36 is the lifecycle/core control plane for JetStream streams: stream creation, assignment, config validation/update, purge/delete flows, mirror/source setup and message processing, and internal subscriptions. This batch is a gating dependency for Batch 37 (Stream Messages) and Batch 38 (Consumer Lifecycle), so fake progress here creates downstream breakage.

The current .NET surface is materially under-ported for this area:

• Batch 36 features are all deferred.
• Most mapped methods are missing from source classes.
• Existing ImplBacklog tests for related classes are largely placeholder-style and need replacement with behavior checks.

Context Findings

Required command outputs

• /usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- batch show 36 --db porting.db
  • Status: pending
  • Features: 92 (3196-3293 with gaps)
  • Tests: 53
  • Depends on: 8,11,12,13,14,15,28
  • Go file: server/stream.go
• /usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- batch list --db porting.db
  • Confirms chain: 36 -> 37,38
• /usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- report summary --db porting.db
  • Overall progress: 1924/6942 (27.7%)

Environment note: dotnet is not on PATH in this shell. Use /usr/local/share/dotnet/dotnet explicitly.

Feature ownership split (from porting.db)

• NatsStream: 76
• Account: 4
• PersistModeType: 3
• StreamSourceInfo: 2
• StreamSource: 2
• JsAccount: 2
• NatsServer: 1
• InMsg: 1
• ExternalStream: 1

Test split (from porting.db)

• JetStreamEngineTests: 35
• NatsConsumerTests: 4
• JetStreamClusterTests1: 3
• JetStreamClusterTests2: 4
• JetStreamClusterTests3: 3
• ConcurrencyTests1: 2
• JetStreamFileStoreTests: 1
• StorageEngineTests: 1

Additional finding: JetStreamClusterTests1.Impltests.cs and JetStreamClusterTests3.Impltests.cs do not currently exist and must be created.

Approaches

Approach A: Single-file port in existing files only

Port all methods directly into existing files (NatsStream.cs, Account.cs, JetStreamTypes.cs, etc.) without structural splits.

• Pros: minimal file churn.
• Cons: high merge risk, hard reviewability, and poor fault isolation for 76 NatsStream methods.

Approach B (Recommended): Lifecycle-focused partial-file decomposition + staged verification

Keep mapped owning classes intact but split implementation by concern into new partial files for stream lifecycle/mirror/source/subscription paths. Execute in bounded feature groups and test waves with strict evidence gates.

• Pros: manageable review units, clearer ownership, better checkpointing and rollback, aligns with anti-stub guardrails.
• Cons: requires adding partial declarations where classes are currently non-partial.

Approach C: Test-first broad wave before feature completion

Attempt to port all 53 tests up front to drive implementation.

• Pros: fast behavior pressure.
• Cons: most tests depend on missing lifecycle internals and will churn heavily; expensive red-state noise.

Decision: Approach B.

Proposed Design

1. Code organization strategy

Use concern-oriented partial files while preserving mapped class ownership:

• dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.cs (retain core type; convert to partial)
• Create dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.Lifecycle.cs
• Create dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.Mirror.cs
• Create dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.Source.cs
• Create dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.Subscriptions.cs
• dotnet/src/ZB.MOM.NatsNet.Server/Accounts/Account.cs (convert to partial)
• Create dotnet/src/ZB.MOM.NatsNet.Server/Accounts/Account.StreamLifecycle.cs
• dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamTypes.cs (for JsAccount, convert class to partial if split)
• Create dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JsAccount.StreamLifecycle.cs
• dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.cs or NatsServer.*.cs partial for CheckStreamCfg
• dotnet/src/ZB.MOM.NatsNet.Server/JetStream/StoreTypes.cs and StreamTypes.cs for PersistModeType, ExternalStream.Domain, StreamSource/StreamSourceInfo, InMsg

2. Functional decomposition

• Config and creation path: PersistModeType, ExternalStream.Domain, Account.AddStream*, StreamSource.ComposeIName/SetIndexName, NatsServer.CheckStreamCfg, JsAccount.ConfigUpdateCheck, NatsStream.Update*.
• State/advisory path: leader checks, sequence counters (CLFS), created time, create/update/delete advisories, purge/delete/erase behavior.
• Mirror path: mirror setup/retry/cancel, inbound mirror flow-control handling, lag tracking, retry backoff.
• Source path: source setup/retry/cancel, shared source message pump, source headers, start-sequence reconstruction.
• Subscription/internal path: stream/direct/mirror-direct subscriptions, source consumer stop/unsubscribe, batching cleanup, internal subscribe helpers.

3. Test design

Port test IDs in waves tied to implemented behavior:

• Wave 1: file store + cluster/meta + consumer API basics (15 tests)
• Wave 2-4: JetStreamEngineTests heavy stream lifecycle scenarios (12/12/14)

Test files:

• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamEngineTests.Impltests.cs
• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/NatsConsumerTests.Impltests.cs
• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests2.Impltests.cs
• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamFileStoreTests.Impltests.cs
• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests1.Impltests.cs
• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/StorageEngineTests.Impltests.cs
• Create: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests1.Impltests.cs
• Create: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests3.Impltests.cs

4. Verification and risk controls (design-level)

• Per-feature and per-test loops are mandatory before status promotion.
• Stub scans are mandatory after each loop and at each checkpoint.
• Build and test gates are mandatory before every status batch update.
• Status updates are chunked to max 15 IDs.
• If blocked: remain deferred with explicit reason, never placeholder implementation.

Feature Grouping (for implementation plan)

• Group A (20): IDs 3196-3219 (actual mapped IDs, 20 total)
• Group B (20): IDs 3220-3240 (actual mapped IDs, 20 total)
• Group C (20): IDs 3241-3261 (actual mapped IDs, 20 total)
• Group D (20): IDs 3262-3281 (actual mapped IDs, 20 total)
• Group E (12): IDs 3282-3293 (actual mapped IDs, 12 total)

Constraints

• Planning only in this session; no implementation execution.
• Must preserve .NET standards (xUnit 3, Shouldly, NSubstitute, nullable, naming conventions).
• Must avoid fake-pass tests and placeholder feature bodies.

Non-Goals

• Executing Batch 36 implementation in this session.
• Expanding scope beyond Batch 36 mapped features/tests.
• Building new integration infrastructure not required by mapped unit tests.