docs: add final remaining jetstream parity design

This commit is contained in:
Joseph Doherty
2026-02-23 10:28:10 -05:00
parent ee6809aedc
commit cc188fa84d

View File

@@ -0,0 +1,107 @@
# JetStream Final Remaining Parity Design
**Date:** 2026-02-23
**Status:** Approved
**Scope:** Complete all remaining JetStream functionality and required transport prerequisites from Go in the .NET server, with strict parity closure criteria.
## 1. Architecture and Scope Boundary
Remaining parity is executed in three ordered layers:
1. Cluster transport prerequisites
- Complete route wire protocol behavior (RS+/RS-, RMSG forwarding, route pool baseline).
- Replace gateway and leaf-node stub behavior with functional networking/handshake/interest propagation sufficient for JetStream parity dependencies.
2. JetStream semantic completion
- Finish stream/consumer behavior still marked partial (retention/discard/runtime policy enforcement, delivery/replay/fetch/ack edge semantics, dedupe window/expected-header semantics, snapshot/restore durability semantics).
3. Parity closure and verification
- Remove JetStream "partial" status from `differences.md` unless an explicit non-JetStream blocker remains.
- Maintain evidence mapping from Go feature to .NET implementation and proving tests.
## 2. Component Plan
### A. Route/Gateway/Leaf transport completion
- Expand:
- `src/NATS.Server/Routes/RouteManager.cs`
- `src/NATS.Server/Routes/RouteConnection.cs`
- Add wire-level RS+/RS- and RMSG behavior, and route pool baseline behavior.
- Replace stub-only behavior in:
- `src/NATS.Server/Gateways/GatewayManager.cs`
- `src/NATS.Server/LeafNodes/LeafNodeManager.cs`
with functional baseline networking/handshake and interest propagation.
### B. JetStream API surface completion
- Expand:
- `src/NATS.Server/JetStream/Api/JetStreamApiSubjects.cs`
- `src/NATS.Server/JetStream/Api/JetStreamApiRouter.cs`
- `src/NATS.Server/JetStream/Api/Handlers/*`
- Cover remaining Go API families and durable/create/control variants with contract-accurate response shapes.
### C. Stream/consumer semantic completion
- Refine:
- `src/NATS.Server/JetStream/StreamManager.cs`
- `src/NATS.Server/JetStream/ConsumerManager.cs`
- `src/NATS.Server/JetStream/Consumers/*`
- `src/NATS.Server/JetStream/Publish/*`
- Ensure modeled policies are fully enforced at runtime.
### D. Store/recovery and RAFT semantics
- Expand:
- `src/NATS.Server/JetStream/Storage/*`
- `src/NATS.Server/JetStream/Snapshots/*`
- `src/NATS.Server/Raft/*`
- Move from shape-level support to behavior-level durability and distributed-state correctness.
### E. Monitoring + evidence
- Update JetStream/cluster monitoring paths and models to reflect real runtime behavior.
- Keep parity map and `differences.md` synchronized with test-backed implementation state.
## 3. Data Flow and Behavioral Contracts
1. Inter-server flow
- Subscription changes emit RS+/RS- over links.
- Remote interest updates local routing state.
- Publish with remote interest forwards via RMSG-equivalent behavior preserving subject/reply/header semantics.
2. JetStream API flow
- `$JS.API.*` arrives through request/reply message path.
- Router dispatches handlers; handlers validate contract and cluster state; responses encode deterministic success/error semantics.
3. Publish/capture flow
- Publish traverses normal routing and JetStream capture.
- Preconditions run before append.
- Append mutates stream state/indexes; mirror/source/consumer engines observe committed state.
4. Consumer delivery flow
- Pull/push share canonical pending/ack/redelivery state.
- Control operations (`pause/reset/unpin/request-next/update`) mutate consistent state.
- Restart/recovery preserves deterministic observable behavior.
5. Store/recovery flow
- Writes update payload and lookup structures.
- Snapshot/restore and restart replay reconstruct equivalent stream/consumer state.
- RAFT gates cluster-visible transitions where required.
6. Observability flow
- `/jsz`, `/varz`, and cluster endpoints report live behavior after transport/JetStream completion.
## 4. Error Handling and Verification
### Error handling
- Preserve JetStream-specific error semantics over generic fallbacks.
- Maintain separate error classes for:
- request/config validation
- state conflicts/not found
- leadership/quorum/stepdown
- transport connectivity/peer state
- storage/recovery integrity.
### Strict completion gate
1. No remaining JetStream "partial" in `differences.md` unless clearly blocked outside JetStream scope and explicitly documented.
2. Unit and integration evidence for each completed feature area (transport + JetStream).
3. Parity mapping document updated with Go contract, .NET file paths, proving tests, and status.
4. Regression gates pass:
- focused JetStream/route/gateway/leaf/raft suites
- full `dotnet test`
- verification summary artifact.