# Full JetStream and Cluster Prerequisite Parity Design

**Date:** 2026-02-23  
**Status:** Approved  
**Scope:** Port JetStream from Go with all prerequisite subsystems required for full Go JetStream test parity, including cluster route/gateway/leaf behaviors and RAFT/meta-cluster semantics.  
**Verification Gate:** Go JetStream-focused test suites in `golang/nats-server/server/` plus new/updated .NET tests.  
**Cutover Model:** Single end-to-end cutover (no interim acceptance gates).

## 1. Architecture

The implementation uses a full in-process .NET parity architecture that mirrors Go subsystem boundaries while keeping strict internal contracts.

1. Core Server Layer (`NatsServer`/`NatsClient`)
- Extend existing server/client runtime to support full client kinds and inter-server protocol paths.
- Preserve responsibility for socket lifecycle, parser integration, auth entry, and local dispatch.

2. Cluster Fabric Layer
- Add route mesh, gateway links, leafnode links, interest propagation, and remote subscription accounting.
- Provide transport-neutral contracts consumed by JetStream and RAFT replication services.

3. JetStream Control Plane
- Add account-scoped JetStream managers, API subject handlers (`$JS.API.*`), stream/consumer metadata lifecycle, advisories, and limit enforcement.
- Integrate with RAFT/meta services for replicated decisions.

4. JetStream Data Plane
- Add stream ingest path, retention/eviction logic, consumer delivery/ack/redelivery, mirror/source orchestration, and flow-control behavior.
- Use pluggable storage abstractions with parity-focused behavior.

5. RAFT and Replication Layer
- Implement meta-group plus per-asset replication groups, election/term logic, log replication, snapshots, and catchup.
- Expose deterministic commit/applied hooks to JetStream runtime layers.

6. Storage Layer
- Implement memstore and filestore with sequence indexing, subject indexing, compaction/snapshot support, and recovery semantics.

7. Observability Layer
- Upgrade `/jsz` and `/varz` JetStream blocks from placeholders to live runtime reporting with Go-compatible response shape.

## 2. Components and Contracts

### 2.1 New component families

1. Cluster and interserver subsystem
- Add route/gateway/leaf and interserver protocol operations under `src/NATS.Server/`.
- Extend parser/dispatcher with route/leaf/account operations currently excluded.
- Expand client-kind model and command routing constraints.

2. JetStream API and domain model
- Add `src/NATS.Server/JetStream/` subtree for API payload models, stream/consumer models, and error templates/codes.

3. JetStream runtime
- Add stream manager, consumer manager, ack processor, delivery scheduler, mirror/source orchestration, and flow control handlers.
- Integrate publish path with stream capture/store/ack behavior.

4. RAFT subsystem
- Add `src/NATS.Server/Raft/` for replicated logs, elections, snapshots, and membership operations.

5. Storage subsystem
- Add `src/NATS.Server/JetStream/Storage/` for `MemStore` and `FileStore`, sequence/subject indexes, and restart recovery.

### 2.2 Existing components to upgrade

1. `src/NATS.Server/NatsOptions.cs`
- Add full config surface for clustering, JetStream, storage, placement, and parity-required limits.

2. `src/NATS.Server/Configuration/ConfigProcessor.cs`
- Replace silent ignore behavior for cluster/jetstream keys with parsing, mapping, and validation.

3. `src/NATS.Server/Protocol/NatsParser.cs` and `src/NATS.Server/NatsClient.cs`
- Add missing interserver operations and kind-aware dispatch paths needed for clustered JetStream behavior.

4. Monitoring components
- Upgrade `src/NATS.Server/Monitoring/MonitorServer.cs` and `src/NATS.Server/Monitoring/Varz.cs`.
- Add/extend JS monitoring handlers and models for `/jsz` and JetStream runtime fields.

## 3. Data Flow and Behavioral Semantics

1. Inbound publish path
- Parse client publish commands, apply auth/permission checks, route to local subscribers and JetStream candidates.
- For JetStream subjects: apply preconditions, append to store, replicate via RAFT (as required), apply committed state, return Go-compatible pub ack.

2. Consumer delivery path
- Use shared push/pull state model for pending, ack floor, redelivery timers, flow control, and max ack pending.
- Enforce retention policy semantics (limits/interest/workqueue), filter subject behavior, replay policy, and eviction behavior.

3. Replication and control flow
- Meta RAFT governs replicated metadata decisions.
- Per-stream/per-consumer groups replicate state and snapshots.
- Leader changes preserve at-least-once delivery and consumer state invariants.

4. Recovery flow
- Reconstruct stream/consumer/store state on startup.
- In clustered mode, rejoin replication groups and catch up before serving full API/delivery workload.
- Preserve sequence continuity, subject indexes, delete markers, and pending/redelivery state.

5. Monitoring flow
- `/varz` JetStream fields and `/jsz` return live runtime state.
- Advisory and metric surfaces update from control-plane and data-plane events.

## 4. Error Handling and Operational Constraints

1. API error parity
- Match canonical JetStream codes/messages for validation failures, state conflicts, limits, leadership/quorum issues, and storage failures.

2. Protocol behavior
- Preserve normal client compatibility while adding interserver protocol and internal client-kind restrictions.

3. Storage and consistency failures
- Classify corruption/truncation/checksum/snapshot failures as recoverable vs non-recoverable.
- Avoid silent data loss and emit monitoring/advisory signals where parity requires.

4. Cluster and RAFT fault handling
- Explicitly handle no-quorum, stale leader, delayed apply, peer removal, catchup lag, and stepdown transitions.
- Return leadership-aware API errors.

5. Config/reload behavior
- Treat JetStream and cluster config as first-class with strict validation.
- Mirror Go-like reloadable vs restart-required change boundaries.

## 5. Testing and Verification Strategy

1. .NET unit tests
- Add focused tests for JetStream API validation, stream and consumer state, RAFT primitives, mem/file store invariants, and config parsing/validation.

2. .NET integration tests
- Add end-to-end tests for publish/store/consume/ack behavior, retention policies, restart recovery, and clustered prerequisites used by JetStream.

3. Parity harness
- Maintain mapping of Go JetStream test categories to .NET feature areas.
- Execute JetStream-focused Go tests from `golang/nats-server/server/` as acceptance benchmark.

4. `differences.md` policy
- Update only after verification gate passes.
- Remove opening JetStream exclusion scope statement and replace with updated parity scope.

## 6. Scope Decisions Captured

- Include all prerequisite non-JetStream subsystems required to satisfy full Go JetStream tests.
- Verification target is full Go JetStream-focused parity, not a narrowed subset.
- Delivery model is single end-to-end cutover.
- `differences.md` top-level scope statement will be updated to include JetStream and clustering parity coverage once verified.
