104 lines
5.0 KiB
Markdown
104 lines
5.0 KiB
Markdown
# JetStream Remaining Parity Design
|
|
|
|
**Date:** 2026-02-23
|
|
**Status:** Approved
|
|
**Scope:** Identify and port all remaining JetStream functionality from Go to .NET, including missing API surface, runtime behaviors, storage/recovery semantics, and cluster/RAFT control operations.
|
|
**Verification Mode:** Dual gate — expanded .NET unit/integration evidence plus maintained Go-to-.NET parity mapping.
|
|
|
|
## 1. Architecture and Parity Boundary
|
|
|
|
The existing .NET JetStream implementation is treated as a bootstrap. Remaining work completes parity across five layers:
|
|
|
|
1. JetStream API Surface Layer
|
|
- Expand `$JS.API.*` handling from current minimal subset to remaining stream, consumer, direct, account, meta, and server operation families.
|
|
- Add response/error contracts with Go-compatible behavior for each operation family.
|
|
|
|
2. JetStream Domain Runtime Layer
|
|
- Upgrade stream and consumer state machines to support remaining lifecycle operations and behavior contracts.
|
|
- Centralize state transitions so all API handlers use shared domain logic.
|
|
|
|
3. Storage and Recovery Layer
|
|
- Extend mem/file stores for remaining retrieval, purge/delete, indexing, and recovery/snapshot semantics.
|
|
- Ensure deterministic state reconstruction across restart and restore scenarios.
|
|
|
|
4. Cluster and RAFT Consistency Layer
|
|
- Upgrade simplified RAFT/meta/replica behavior to support remaining control-plane operations and failure semantics.
|
|
- Keep interface seams explicit between JetStream runtime and replication internals.
|
|
|
|
5. Verification and Parity Mapping Layer
|
|
- Maintain a feature-level Go-to-.NET map for API/behavior/test evidence.
|
|
- Use map + tests as completion criteria for each feature area.
|
|
|
|
## 2. Component Plan
|
|
|
|
### A. API Routing and Contracts
|
|
- Expand `src/NATS.Server/JetStream/Api/JetStreamApiRouter.cs` to route all remaining subject families.
|
|
- Add handlers under `src/NATS.Server/JetStream/Api/Handlers/` for missing stream, consumer, direct, account, meta, and server operations.
|
|
- Expand `src/NATS.Server/JetStream/Api/` response and error contracts to represent remaining operation results.
|
|
|
|
### B. Stream and Consumer Runtime
|
|
- Refactor `src/NATS.Server/JetStream/StreamManager.cs` and `src/NATS.Server/JetStream/ConsumerManager.cs` to support full lifecycle and state semantics for remaining APIs.
|
|
- Expand `src/NATS.Server/JetStream/Models/` for missing state/config domains and policy types.
|
|
|
|
### C. Publish Preconditions and Delivery
|
|
- Extend `src/NATS.Server/JetStream/Publish/` preconditions and ack shaping for remaining contracts.
|
|
- Expand `src/NATS.Server/JetStream/Consumers/` to support remaining request-next, pause/reset/unpin, and redelivery policy semantics.
|
|
|
|
### D. Storage, Snapshot, and Restore
|
|
- Expand `src/NATS.Server/JetStream/Storage/` for missing indexes, retrieval modes, purge/delete variants, snapshot/restore semantics, and consistency checks.
|
|
|
|
### E. Cluster and RAFT Control Plane
|
|
- Upgrade `src/NATS.Server/Raft/` and `src/NATS.Server/JetStream/Cluster/` for remaining leader/peer/remove/move/stepdown control behaviors used by JetStream operations.
|
|
|
|
### F. Test and Evidence Artifacts
|
|
- Add missing test suites in `tests/NATS.Server.Tests/` by API family and behavior family.
|
|
- Maintain parity evidence docs in `docs/plans/` tying Go contracts to .NET implementation and tests.
|
|
|
|
## 3. Data Flow and Behavioral Contracts
|
|
|
|
1. API Request Flow
|
|
- Route subject -> parse/validate -> invoke domain manager -> return typed success/error response.
|
|
- Remove generic fallback responses where Go specifies domain errors.
|
|
|
|
2. Publish-to-Stream Flow
|
|
- Subject resolution, precondition validation, store append, state updates, and ack generation must align with remaining Go contracts.
|
|
|
|
3. Consumer Delivery Flow
|
|
- Pull and push share canonical pending/ack/redelivery model.
|
|
- Control operations (pause/reset/unpin/delete/request-next) mutate the same state model.
|
|
|
|
4. Store and Recovery Flow
|
|
- Writes update both payload and lookup/index state for message retrieval operations.
|
|
- Snapshot/restore/restart paths preserve sequence/state invariants.
|
|
|
|
5. Cluster Control Flow
|
|
- Meta and replica operations enforce leadership/quorum semantics and deterministic error signaling.
|
|
|
|
6. Monitoring and Diagnostics Flow
|
|
- `/jsz` and JetStream `/varz` fields reflect live state for newly implemented features.
|
|
|
|
## 4. Error Handling and Verification
|
|
|
|
### Error Handling
|
|
- Use deterministic JetStream error mapping by failure class:
|
|
- request/config validation
|
|
- not-found/conflict
|
|
- leadership/quorum
|
|
- storage/recovery.
|
|
|
|
### Testing
|
|
- Expand tests beyond smoke coverage to feature-complete suites for:
|
|
- API families
|
|
- stream lifecycle/state
|
|
- consumer lifecycle/ack/redelivery
|
|
- storage/recovery/snapshot
|
|
- RAFT/control operations tied to JetStream.
|
|
|
|
### Dual Gate
|
|
1. .NET test evidence for each newly ported feature.
|
|
2. Parity mapping artifact showing Go contract, .NET implementation location, and proving test.
|
|
|
|
### `differences.md` Update Policy
|
|
- Update JetStream-related entries only after dual gate evidence is complete for remaining scope.
|
|
- Keep explicit notes for any deliberate deferrals.
|