108 lines
4.6 KiB
Markdown
108 lines
4.6 KiB
Markdown
# JetStream Final Remaining Parity Design
|
|
|
|
**Date:** 2026-02-23
|
|
**Status:** Approved
|
|
**Scope:** Complete all remaining JetStream functionality and required transport prerequisites from Go in the .NET server, with strict parity closure criteria.
|
|
|
|
## 1. Architecture and Scope Boundary
|
|
|
|
Remaining parity is executed in three ordered layers:
|
|
|
|
1. Cluster transport prerequisites
|
|
- Complete route wire protocol behavior (RS+/RS-, RMSG forwarding, route pool baseline).
|
|
- Replace gateway and leaf-node stub behavior with functional networking/handshake/interest propagation sufficient for JetStream parity dependencies.
|
|
|
|
2. JetStream semantic completion
|
|
- Finish stream/consumer behavior still marked partial (retention/discard/runtime policy enforcement, delivery/replay/fetch/ack edge semantics, dedupe window/expected-header semantics, snapshot/restore durability semantics).
|
|
|
|
3. Parity closure and verification
|
|
- Remove JetStream "partial" status from `differences.md` unless an explicit non-JetStream blocker remains.
|
|
- Maintain evidence mapping from Go feature to .NET implementation and proving tests.
|
|
|
|
## 2. Component Plan
|
|
|
|
### A. Route/Gateway/Leaf transport completion
|
|
- Expand:
|
|
- `src/NATS.Server/Routes/RouteManager.cs`
|
|
- `src/NATS.Server/Routes/RouteConnection.cs`
|
|
- Add wire-level RS+/RS- and RMSG behavior, and route pool baseline behavior.
|
|
- Replace stub-only behavior in:
|
|
- `src/NATS.Server/Gateways/GatewayManager.cs`
|
|
- `src/NATS.Server/LeafNodes/LeafNodeManager.cs`
|
|
with functional baseline networking/handshake and interest propagation.
|
|
|
|
### B. JetStream API surface completion
|
|
- Expand:
|
|
- `src/NATS.Server/JetStream/Api/JetStreamApiSubjects.cs`
|
|
- `src/NATS.Server/JetStream/Api/JetStreamApiRouter.cs`
|
|
- `src/NATS.Server/JetStream/Api/Handlers/*`
|
|
- Cover remaining Go API families and durable/create/control variants with contract-accurate response shapes.
|
|
|
|
### C. Stream/consumer semantic completion
|
|
- Refine:
|
|
- `src/NATS.Server/JetStream/StreamManager.cs`
|
|
- `src/NATS.Server/JetStream/ConsumerManager.cs`
|
|
- `src/NATS.Server/JetStream/Consumers/*`
|
|
- `src/NATS.Server/JetStream/Publish/*`
|
|
- Ensure modeled policies are fully enforced at runtime.
|
|
|
|
### D. Store/recovery and RAFT semantics
|
|
- Expand:
|
|
- `src/NATS.Server/JetStream/Storage/*`
|
|
- `src/NATS.Server/JetStream/Snapshots/*`
|
|
- `src/NATS.Server/Raft/*`
|
|
- Move from shape-level support to behavior-level durability and distributed-state correctness.
|
|
|
|
### E. Monitoring + evidence
|
|
- Update JetStream/cluster monitoring paths and models to reflect real runtime behavior.
|
|
- Keep parity map and `differences.md` synchronized with test-backed implementation state.
|
|
|
|
## 3. Data Flow and Behavioral Contracts
|
|
|
|
1. Inter-server flow
|
|
- Subscription changes emit RS+/RS- over links.
|
|
- Remote interest updates local routing state.
|
|
- Publish with remote interest forwards via RMSG-equivalent behavior preserving subject/reply/header semantics.
|
|
|
|
2. JetStream API flow
|
|
- `$JS.API.*` arrives through request/reply message path.
|
|
- Router dispatches handlers; handlers validate contract and cluster state; responses encode deterministic success/error semantics.
|
|
|
|
3. Publish/capture flow
|
|
- Publish traverses normal routing and JetStream capture.
|
|
- Preconditions run before append.
|
|
- Append mutates stream state/indexes; mirror/source/consumer engines observe committed state.
|
|
|
|
4. Consumer delivery flow
|
|
- Pull/push share canonical pending/ack/redelivery state.
|
|
- Control operations (`pause/reset/unpin/request-next/update`) mutate consistent state.
|
|
- Restart/recovery preserves deterministic observable behavior.
|
|
|
|
5. Store/recovery flow
|
|
- Writes update payload and lookup structures.
|
|
- Snapshot/restore and restart replay reconstruct equivalent stream/consumer state.
|
|
- RAFT gates cluster-visible transitions where required.
|
|
|
|
6. Observability flow
|
|
- `/jsz`, `/varz`, and cluster endpoints report live behavior after transport/JetStream completion.
|
|
|
|
## 4. Error Handling and Verification
|
|
|
|
### Error handling
|
|
- Preserve JetStream-specific error semantics over generic fallbacks.
|
|
- Maintain separate error classes for:
|
|
- request/config validation
|
|
- state conflicts/not found
|
|
- leadership/quorum/stepdown
|
|
- transport connectivity/peer state
|
|
- storage/recovery integrity.
|
|
|
|
### Strict completion gate
|
|
1. No remaining JetStream "partial" in `differences.md` unless clearly blocked outside JetStream scope and explicitly documented.
|
|
2. Unit and integration evidence for each completed feature area (transport + JetStream).
|
|
3. Parity mapping document updated with Go contract, .NET file paths, proving tests, and status.
|
|
4. Regression gates pass:
|
|
- focused JetStream/route/gateway/leaf/raft suites
|
|
- full `dotnet test`
|
|
- verification summary artifact.
|