natsdotnet/docs/plans/2026-02-23-jetstream-remaining-parity-design.md

JetStream Remaining Parity Design

Date: 2026-02-23
Status: Approved
Scope: Identify and port all remaining JetStream functionality from Go to .NET, including missing API surface, runtime behaviors, storage/recovery semantics, and cluster/RAFT control operations.
Verification Mode: Dual gate — expanded .NET unit/integration evidence plus maintained Go-to-.NET parity mapping.

1. Architecture and Parity Boundary

The existing .NET JetStream implementation is treated as a bootstrap. Remaining work completes parity across five layers:

1. JetStream API Surface Layer

• Expand $JS.API.* handling from current minimal subset to remaining stream, consumer, direct, account, meta, and server operation families.
• Add response/error contracts with Go-compatible behavior for each operation family.

2. JetStream Domain Runtime Layer

• Upgrade stream and consumer state machines to support remaining lifecycle operations and behavior contracts.
• Centralize state transitions so all API handlers use shared domain logic.

3. Storage and Recovery Layer

• Extend mem/file stores for remaining retrieval, purge/delete, indexing, and recovery/snapshot semantics.
• Ensure deterministic state reconstruction across restart and restore scenarios.

4. Cluster and RAFT Consistency Layer

• Upgrade simplified RAFT/meta/replica behavior to support remaining control-plane operations and failure semantics.
• Keep interface seams explicit between JetStream runtime and replication internals.

5. Verification and Parity Mapping Layer

• Maintain a feature-level Go-to-.NET map for API/behavior/test evidence.
• Use map + tests as completion criteria for each feature area.

2. Component Plan

A. API Routing and Contracts

• Expand src/NATS.Server/JetStream/Api/JetStreamApiRouter.cs to route all remaining subject families.
• Add handlers under src/NATS.Server/JetStream/Api/Handlers/ for missing stream, consumer, direct, account, meta, and server operations.
• Expand src/NATS.Server/JetStream/Api/ response and error contracts to represent remaining operation results.

B. Stream and Consumer Runtime

• Refactor src/NATS.Server/JetStream/StreamManager.cs and src/NATS.Server/JetStream/ConsumerManager.cs to support full lifecycle and state semantics for remaining APIs.
• Expand src/NATS.Server/JetStream/Models/ for missing state/config domains and policy types.

C. Publish Preconditions and Delivery

• Extend src/NATS.Server/JetStream/Publish/ preconditions and ack shaping for remaining contracts.
• Expand src/NATS.Server/JetStream/Consumers/ to support remaining request-next, pause/reset/unpin, and redelivery policy semantics.

D. Storage, Snapshot, and Restore

• Expand src/NATS.Server/JetStream/Storage/ for missing indexes, retrieval modes, purge/delete variants, snapshot/restore semantics, and consistency checks.

E. Cluster and RAFT Control Plane

• Upgrade src/NATS.Server/Raft/ and src/NATS.Server/JetStream/Cluster/ for remaining leader/peer/remove/move/stepdown control behaviors used by JetStream operations.

F. Test and Evidence Artifacts

• Add missing test suites in tests/NATS.Server.Tests/ by API family and behavior family.
• Maintain parity evidence docs in docs/plans/ tying Go contracts to .NET implementation and tests.

3. Data Flow and Behavioral Contracts

1. API Request Flow

• Route subject -> parse/validate -> invoke domain manager -> return typed success/error response.
• Remove generic fallback responses where Go specifies domain errors.

2. Publish-to-Stream Flow

• Subject resolution, precondition validation, store append, state updates, and ack generation must align with remaining Go contracts.

3. Consumer Delivery Flow

• Pull and push share canonical pending/ack/redelivery model.
• Control operations (pause/reset/unpin/delete/request-next) mutate the same state model.

4. Store and Recovery Flow

• Writes update both payload and lookup/index state for message retrieval operations.
• Snapshot/restore/restart paths preserve sequence/state invariants.

5. Cluster Control Flow

• Meta and replica operations enforce leadership/quorum semantics and deterministic error signaling.

6. Monitoring and Diagnostics Flow

• /jsz and JetStream /varz fields reflect live state for newly implemented features.

4. Error Handling and Verification

Error Handling

• Use deterministic JetStream error mapping by failure class:
  • request/config validation
  • not-found/conflict
  • leadership/quorum
  • storage/recovery.

Testing

• Expand tests beyond smoke coverage to feature-complete suites for:
  • API families
  • stream lifecycle/state
  • consumer lifecycle/ack/redelivery
  • storage/recovery/snapshot
  • RAFT/control operations tied to JetStream.

Dual Gate

1. .NET test evidence for each newly ported feature.
2. Parity mapping artifact showing Go contract, .NET implementation location, and proving test.

differences.md Update Policy

• Update JetStream-related entries only after dual gate evidence is complete for remaining scope.
• Keep explicit notes for any deliberate deferrals.