From ffb165fd8a904695346918b3710cec4d661c2268 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Mon, 23 Feb 2026 14:25:13 -0500 Subject: [PATCH] docs: add strict full go parity design --- ...02-23-nats-strict-full-go-parity-design.md | 163 ++++++++++++++++++ 1 file changed, 163 insertions(+) create mode 100644 docs/plans/2026-02-23-nats-strict-full-go-parity-design.md diff --git a/docs/plans/2026-02-23-nats-strict-full-go-parity-design.md b/docs/plans/2026-02-23-nats-strict-full-go-parity-design.md new file mode 100644 index 0000000..6bf6879 --- /dev/null +++ b/docs/plans/2026-02-23-nats-strict-full-go-parity-design.md @@ -0,0 +1,163 @@ +# NATS Strict Full Go Parity Design + +**Date:** 2026-02-23 +**Status:** Approved +**Scope:** Identify and close all remaining NATS functionality gaps between Go and .NET using behavior-level parity (not limited to existing parity-table status), with required unit/integration test coverage and parity-doc synchronization. + +## 1. Architecture and Scope Boundary + +### Closure model +Each parity capability closes only when all three are complete: + +1. Behavior +The .NET runtime behavior matches Go contract semantics at protocol and subsystem level. + +2. Tests +Positive and edge/failure tests exist and fail on regression. + +3. Docs +`differences.md` and parity artifacts reflect the validated behavior state. + +### Scope interpretation +This cycle treats remaining parity as capability-level and code-backed, not as a direct read of current table status alone. Existing contradictions between prose claims and observable implementation are treated as parity debt and are in scope. + +### Execution order +1. Inter-server fabric semantics (routes/gateway/leaf, account-scoped interest/delivery) +2. Auth/account/config/monitor operational semantics +3. JetStream runtime state machines +4. JetStream storage durability + RAFT/cluster governance +5. MQTT protocol/transport parity beyond current baseline +6. Documentation and evidence synchronization + +## 2. Component Plan + +### A. Inter-server fabric parity +Primary files: +- `src/NATS.Server/NatsServer.cs` +- `src/NATS.Server/Routes/RouteConnection.cs` +- `src/NATS.Server/Routes/RouteManager.cs` +- `src/NATS.Server/Gateways/GatewayConnection.cs` +- `src/NATS.Server/Gateways/GatewayManager.cs` +- `src/NATS.Server/LeafNodes/LeafConnection.cs` +- `src/NATS.Server/LeafNodes/LeafNodeManager.cs` +- `src/NATS.Server/Subscriptions/SubList.cs` + +Focus: +- account-correct remote delivery path (not global-only shortcuts) +- idempotent remote interest propagation under reconnect/duplicates +- durable reply-remap and loop-marker transparency rules + +### B. MQTT protocol parity +Primary files: +- `src/NATS.Server/Mqtt/MqttProtocolParser.cs` +- `src/NATS.Server/Mqtt/MqttConnection.cs` +- `src/NATS.Server/Mqtt/MqttListener.cs` +- `src/NATS.Server/MqttOptions.cs` +- `src/NATS.Server/Configuration/ConfigProcessor.cs` + +Focus: +- replace line-oriented pseudo protocol with byte-level MQTT packet parsing/writing +- implement connection/session/subscribe/publish acknowledgement contracts +- align MQTT auth/TLS/keepalive behavior with configured options + +### C. JetStream runtime parity +Primary files: +- `src/NATS.Server/JetStream/JetStreamService.cs` +- `src/NATS.Server/JetStream/StreamManager.cs` +- `src/NATS.Server/JetStream/ConsumerManager.cs` +- `src/NATS.Server/JetStream/Consumers/AckProcessor.cs` +- `src/NATS.Server/JetStream/Consumers/PullConsumerEngine.cs` +- `src/NATS.Server/JetStream/Consumers/PushConsumerEngine.cs` +- `src/NATS.Server/JetStream/MirrorSource/*` + +Focus: +- policy-divergent retention runtime behavior (`Limits/Interest/WorkQueue`) +- deterministic delivery/ack/backoff/replay/flow-control state-machine transitions +- mirror/source behavior under filters, transform, and cross-account paths + +### D. Storage + RAFT + governance parity +Primary files: +- `src/NATS.Server/JetStream/Storage/FileStore.cs` +- `src/NATS.Server/JetStream/Storage/MemStore.cs` +- `src/NATS.Server/Raft/RaftNode.cs` +- `src/NATS.Server/Raft/RaftReplicator.cs` +- `src/NATS.Server/Raft/RaftTransport.cs` +- `src/NATS.Server/Raft/RaftSnapshotStore.cs` +- `src/NATS.Server/JetStream/Cluster/JetStreamMetaGroup.cs` +- `src/NATS.Server/JetStream/Cluster/StreamReplicaGroup.cs` + +Focus: +- durability invariants across restart/prune/recovery cycles +- term/quorum/next-index/snapshot/membership semantics with convergence behavior +- meta/replica governance actions reflected in committed state and routing outcomes + +### E. Operational parity surfaces +Primary files: +- `src/NATS.Server/Monitoring/MonitorServer.cs` +- `src/NATS.Server/Monitoring/PprofHandler.cs` +- `src/NATS.Server/Configuration/ConfigReloader.cs` +- `src/NATS.Server/NatsOptions.cs` +- `src/NATS.Server/Events/InternalEventSystem.cs` + +Focus: +- replace synthetic profiling behavior with parity-meaningful runtime profiling outputs +- close option-surface vs runtime-semantics drift for key advanced options +- align event/monitor behavior with capability claims + +### F. Documentation and evidence +Primary files: +- `differences.md` +- `docs/plans/2026-02-23-jetstream-remaining-parity-map.md` +- `docs/plans/2026-02-23-jetstream-remaining-parity-verification.md` + +Focus: +- remove contradictory parity claims +- ensure every claimed closure maps to concrete behavior and test evidence + +## 3. Data Flow and Behavioral Contracts + +1. Inter-server delivery contract +- route/gateway/leaf inbound delivery resolves with correct account scope and permissions +- reply-remap and leaf-loop markers remain transport-internal and reversible +- remote interest convergence stays idempotent across reconnect churn + +2. MQTT protocol contract +- fixed-header/remaining-length packet framing must be valid on read/write paths +- QoS/session/ack transitions must be deterministic and stateful +- auth and account association must fail closed on invalid credentials or malformed packets + +3. JetStream stream contract +- retention modes diverge by runtime behavior, not only config parsing +- preconditions and policy guards execute before irreversible storage changes +- max-age/per-subject/dedupe behavior remains consistent under load and restart + +4. JetStream consumer contract +- deliver policy cursor initialization and progression are deterministic +- ack/redelivery/backoff/max-deliver transitions preserve monotonic pending/floor invariants +- push/pull flows maintain consistent flow-control and replay timing contracts + +5. Storage/RAFT/governance contract +- stream/store state invariants remain stable after prune/recovery/snapshot cycles +- commit visibility is quorum-gated with stale-term and mismatch handling +- governance actions (stepdown/placement/cluster forwarding) reflect committed consensus state + +## 4. Error Handling, Testing Strategy, and Completion Gates + +### Error handling +1. Preserve protocol-specific error surfaces across NATS, JetStream, and MQTT. +2. Fail closed on account-scoping, authorization, and consensus-precondition violations. +3. Prevent partial mutation on multi-step operations (publish+replicate, control+governance). + +### Testing strategy +1. Test-first per capability batch (failing test first, then minimal implementation). +2. Each capability requires positive + edge/failure coverage. +3. Multi-node, transport, RAFT, and MQTT protocol semantics use integration tests; low-level state transitions use focused unit tests. +4. No parity claim is accepted from docs-only or hook-only evidence. + +### Completion gates +1. Capability closes only with Behavior + Tests + Docs complete. +2. Full verification gate must pass: +- focused suites for each completed batch +- `dotnet test -v minimal` +3. `differences.md` may only claim closure where behavior is demonstrably ported. +4. Deferred parity items are allowed only with explicit blocker rationale and observable evidence.