diff --git a/docs/plans/2026-02-23-jetstream-remaining-parity-design.md b/docs/plans/2026-02-23-jetstream-remaining-parity-design.md new file mode 100644 index 0000000..ba9460c --- /dev/null +++ b/docs/plans/2026-02-23-jetstream-remaining-parity-design.md @@ -0,0 +1,103 @@ +# JetStream Remaining Parity Design + +**Date:** 2026-02-23 +**Status:** Approved +**Scope:** Identify and port all remaining JetStream functionality from Go to .NET, including missing API surface, runtime behaviors, storage/recovery semantics, and cluster/RAFT control operations. +**Verification Mode:** Dual gate — expanded .NET unit/integration evidence plus maintained Go-to-.NET parity mapping. + +## 1. Architecture and Parity Boundary + +The existing .NET JetStream implementation is treated as a bootstrap. Remaining work completes parity across five layers: + +1. JetStream API Surface Layer +- Expand `$JS.API.*` handling from current minimal subset to remaining stream, consumer, direct, account, meta, and server operation families. +- Add response/error contracts with Go-compatible behavior for each operation family. + +2. JetStream Domain Runtime Layer +- Upgrade stream and consumer state machines to support remaining lifecycle operations and behavior contracts. +- Centralize state transitions so all API handlers use shared domain logic. + +3. Storage and Recovery Layer +- Extend mem/file stores for remaining retrieval, purge/delete, indexing, and recovery/snapshot semantics. +- Ensure deterministic state reconstruction across restart and restore scenarios. + +4. Cluster and RAFT Consistency Layer +- Upgrade simplified RAFT/meta/replica behavior to support remaining control-plane operations and failure semantics. +- Keep interface seams explicit between JetStream runtime and replication internals. + +5. Verification and Parity Mapping Layer +- Maintain a feature-level Go-to-.NET map for API/behavior/test evidence. +- Use map + tests as completion criteria for each feature area. + +## 2. Component Plan + +### A. API Routing and Contracts +- Expand `src/NATS.Server/JetStream/Api/JetStreamApiRouter.cs` to route all remaining subject families. +- Add handlers under `src/NATS.Server/JetStream/Api/Handlers/` for missing stream, consumer, direct, account, meta, and server operations. +- Expand `src/NATS.Server/JetStream/Api/` response and error contracts to represent remaining operation results. + +### B. Stream and Consumer Runtime +- Refactor `src/NATS.Server/JetStream/StreamManager.cs` and `src/NATS.Server/JetStream/ConsumerManager.cs` to support full lifecycle and state semantics for remaining APIs. +- Expand `src/NATS.Server/JetStream/Models/` for missing state/config domains and policy types. + +### C. Publish Preconditions and Delivery +- Extend `src/NATS.Server/JetStream/Publish/` preconditions and ack shaping for remaining contracts. +- Expand `src/NATS.Server/JetStream/Consumers/` to support remaining request-next, pause/reset/unpin, and redelivery policy semantics. + +### D. Storage, Snapshot, and Restore +- Expand `src/NATS.Server/JetStream/Storage/` for missing indexes, retrieval modes, purge/delete variants, snapshot/restore semantics, and consistency checks. + +### E. Cluster and RAFT Control Plane +- Upgrade `src/NATS.Server/Raft/` and `src/NATS.Server/JetStream/Cluster/` for remaining leader/peer/remove/move/stepdown control behaviors used by JetStream operations. + +### F. Test and Evidence Artifacts +- Add missing test suites in `tests/NATS.Server.Tests/` by API family and behavior family. +- Maintain parity evidence docs in `docs/plans/` tying Go contracts to .NET implementation and tests. + +## 3. Data Flow and Behavioral Contracts + +1. API Request Flow +- Route subject -> parse/validate -> invoke domain manager -> return typed success/error response. +- Remove generic fallback responses where Go specifies domain errors. + +2. Publish-to-Stream Flow +- Subject resolution, precondition validation, store append, state updates, and ack generation must align with remaining Go contracts. + +3. Consumer Delivery Flow +- Pull and push share canonical pending/ack/redelivery model. +- Control operations (pause/reset/unpin/delete/request-next) mutate the same state model. + +4. Store and Recovery Flow +- Writes update both payload and lookup/index state for message retrieval operations. +- Snapshot/restore/restart paths preserve sequence/state invariants. + +5. Cluster Control Flow +- Meta and replica operations enforce leadership/quorum semantics and deterministic error signaling. + +6. Monitoring and Diagnostics Flow +- `/jsz` and JetStream `/varz` fields reflect live state for newly implemented features. + +## 4. Error Handling and Verification + +### Error Handling +- Use deterministic JetStream error mapping by failure class: + - request/config validation + - not-found/conflict + - leadership/quorum + - storage/recovery. + +### Testing +- Expand tests beyond smoke coverage to feature-complete suites for: + - API families + - stream lifecycle/state + - consumer lifecycle/ack/redelivery + - storage/recovery/snapshot + - RAFT/control operations tied to JetStream. + +### Dual Gate +1. .NET test evidence for each newly ported feature. +2. Parity mapping artifact showing Go contract, .NET implementation location, and proving test. + +### `differences.md` Update Policy +- Update JetStream-related entries only after dual gate evidence is complete for remaining scope. +- Keep explicit notes for any deliberate deferrals.