docs: add final remaining jetstream parity design
This commit is contained in:
107
docs/plans/2026-02-23-jetstream-final-remaining-design.md
Normal file
107
docs/plans/2026-02-23-jetstream-final-remaining-design.md
Normal file
@@ -0,0 +1,107 @@
|
||||
# JetStream Final Remaining Parity Design
|
||||
|
||||
**Date:** 2026-02-23
|
||||
**Status:** Approved
|
||||
**Scope:** Complete all remaining JetStream functionality and required transport prerequisites from Go in the .NET server, with strict parity closure criteria.
|
||||
|
||||
## 1. Architecture and Scope Boundary
|
||||
|
||||
Remaining parity is executed in three ordered layers:
|
||||
|
||||
1. Cluster transport prerequisites
|
||||
- Complete route wire protocol behavior (RS+/RS-, RMSG forwarding, route pool baseline).
|
||||
- Replace gateway and leaf-node stub behavior with functional networking/handshake/interest propagation sufficient for JetStream parity dependencies.
|
||||
|
||||
2. JetStream semantic completion
|
||||
- Finish stream/consumer behavior still marked partial (retention/discard/runtime policy enforcement, delivery/replay/fetch/ack edge semantics, dedupe window/expected-header semantics, snapshot/restore durability semantics).
|
||||
|
||||
3. Parity closure and verification
|
||||
- Remove JetStream "partial" status from `differences.md` unless an explicit non-JetStream blocker remains.
|
||||
- Maintain evidence mapping from Go feature to .NET implementation and proving tests.
|
||||
|
||||
## 2. Component Plan
|
||||
|
||||
### A. Route/Gateway/Leaf transport completion
|
||||
- Expand:
|
||||
- `src/NATS.Server/Routes/RouteManager.cs`
|
||||
- `src/NATS.Server/Routes/RouteConnection.cs`
|
||||
- Add wire-level RS+/RS- and RMSG behavior, and route pool baseline behavior.
|
||||
- Replace stub-only behavior in:
|
||||
- `src/NATS.Server/Gateways/GatewayManager.cs`
|
||||
- `src/NATS.Server/LeafNodes/LeafNodeManager.cs`
|
||||
with functional baseline networking/handshake and interest propagation.
|
||||
|
||||
### B. JetStream API surface completion
|
||||
- Expand:
|
||||
- `src/NATS.Server/JetStream/Api/JetStreamApiSubjects.cs`
|
||||
- `src/NATS.Server/JetStream/Api/JetStreamApiRouter.cs`
|
||||
- `src/NATS.Server/JetStream/Api/Handlers/*`
|
||||
- Cover remaining Go API families and durable/create/control variants with contract-accurate response shapes.
|
||||
|
||||
### C. Stream/consumer semantic completion
|
||||
- Refine:
|
||||
- `src/NATS.Server/JetStream/StreamManager.cs`
|
||||
- `src/NATS.Server/JetStream/ConsumerManager.cs`
|
||||
- `src/NATS.Server/JetStream/Consumers/*`
|
||||
- `src/NATS.Server/JetStream/Publish/*`
|
||||
- Ensure modeled policies are fully enforced at runtime.
|
||||
|
||||
### D. Store/recovery and RAFT semantics
|
||||
- Expand:
|
||||
- `src/NATS.Server/JetStream/Storage/*`
|
||||
- `src/NATS.Server/JetStream/Snapshots/*`
|
||||
- `src/NATS.Server/Raft/*`
|
||||
- Move from shape-level support to behavior-level durability and distributed-state correctness.
|
||||
|
||||
### E. Monitoring + evidence
|
||||
- Update JetStream/cluster monitoring paths and models to reflect real runtime behavior.
|
||||
- Keep parity map and `differences.md` synchronized with test-backed implementation state.
|
||||
|
||||
## 3. Data Flow and Behavioral Contracts
|
||||
|
||||
1. Inter-server flow
|
||||
- Subscription changes emit RS+/RS- over links.
|
||||
- Remote interest updates local routing state.
|
||||
- Publish with remote interest forwards via RMSG-equivalent behavior preserving subject/reply/header semantics.
|
||||
|
||||
2. JetStream API flow
|
||||
- `$JS.API.*` arrives through request/reply message path.
|
||||
- Router dispatches handlers; handlers validate contract and cluster state; responses encode deterministic success/error semantics.
|
||||
|
||||
3. Publish/capture flow
|
||||
- Publish traverses normal routing and JetStream capture.
|
||||
- Preconditions run before append.
|
||||
- Append mutates stream state/indexes; mirror/source/consumer engines observe committed state.
|
||||
|
||||
4. Consumer delivery flow
|
||||
- Pull/push share canonical pending/ack/redelivery state.
|
||||
- Control operations (`pause/reset/unpin/request-next/update`) mutate consistent state.
|
||||
- Restart/recovery preserves deterministic observable behavior.
|
||||
|
||||
5. Store/recovery flow
|
||||
- Writes update payload and lookup structures.
|
||||
- Snapshot/restore and restart replay reconstruct equivalent stream/consumer state.
|
||||
- RAFT gates cluster-visible transitions where required.
|
||||
|
||||
6. Observability flow
|
||||
- `/jsz`, `/varz`, and cluster endpoints report live behavior after transport/JetStream completion.
|
||||
|
||||
## 4. Error Handling and Verification
|
||||
|
||||
### Error handling
|
||||
- Preserve JetStream-specific error semantics over generic fallbacks.
|
||||
- Maintain separate error classes for:
|
||||
- request/config validation
|
||||
- state conflicts/not found
|
||||
- leadership/quorum/stepdown
|
||||
- transport connectivity/peer state
|
||||
- storage/recovery integrity.
|
||||
|
||||
### Strict completion gate
|
||||
1. No remaining JetStream "partial" in `differences.md` unless clearly blocked outside JetStream scope and explicitly documented.
|
||||
2. Unit and integration evidence for each completed feature area (transport + JetStream).
|
||||
3. Parity mapping document updated with Go contract, .NET file paths, proving tests, and status.
|
||||
4. Regression gates pass:
|
||||
- focused JetStream/route/gateway/leaf/raft suites
|
||||
- full `dotnet test`
|
||||
- verification summary artifact.
|
||||
Reference in New Issue
Block a user