dohertj2/natsdotnet
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Fix E2E test gaps and add comprehensive E2E + parity test suites

Browse Source 
- Fix pull consumer fetch: send original stream subject in HMSG (not inbox)
  so NATS client distinguishes data messages from control messages
- Fix MaxAge expiry: add background timer in StreamManager for periodic pruning
- Fix JetStream wire format: Go-compatible anonymous objects with string enums,
  proper offset-based pagination for stream/consumer list APIs
- Add 42 E2E black-box tests (core messaging, auth, TLS, accounts, JetStream)
- Add ~1000 parity tests across all subsystems (gaps closure)
- Update gap inventory docs to reflect implementation status
This commit is contained in:
Joseph Doherty
2026-03-12 14:09:23 -04:00
parent 79c1ee8776
commit c30e67a69d
226 changed files with 17801 additions and 709 deletions
Download Patch File Download Diff File Expand all files Collapse all files

62
gaps/raft.md
View File

@@ -91,12 +91,12 @@ Add rows to the Gap Inventory table below. Group by Go source file. Include the
| RaftNode (interface) | raft.go:40-92 | PARTIAL | src/NATS.Server/Raft/IRaftNode.cs:5 | Interface declared but empty — none of the 40+ methods from Go are defined |
| RaftNodeCheckpoint (interface) | raft.go:98-103 | PARTIAL | src/NATS.Server/Raft/RaftSnapshotCheckpoint.cs:7 | Chunk assembly exists but LoadLastSnapshot, AppendEntriesSeq, Abort, InstallSnapshot not matching Go's interface contract |
| WAL (interface) | raft.go:105-118 | PARTIAL | src/NATS.Server/Raft/RaftWal.cs:20 | .NET RaftWal is a concrete file-based WAL; does not implement Go's WAL interface (StoreMsg, LoadMsg, RemoveMsg, Compact, Purge, Truncate, State, FastState, Stop, Delete) |
| Peer (struct) | raft.go:120-125 | PARTIAL | src/NATS.Server/Raft/RaftPeerState.cs:7 | .NET has NextIndex/MatchIndex/LastContact/Active but missing Lag and Current fields from Go's Peer export |
| Peer (struct) | raft.go:120-125 | PORTED | src/NATS.Server/Raft/RaftPeerState.cs | Added missing parity fields (`Lag`, `Current`) plus helpers to recalculate lag and refresh current-state from heartbeat/contact timing |
| RaftState (enum) | raft.go:127-135 | PORTED | src/NATS.Server/Raft/RaftState.cs:4 | All four states: Follower, Leader, Candidate, Closed |
| RaftState.String() | raft.go:137-149 | MISSING | — | No .NET ToString override for RaftState enum |
| RaftConfig (struct) | raft.go:301-317 | MISSING | — | No equivalent configuration struct (Name, Store, Log, Track, Observer, Recovering, ScaleUp) |
| CommittedEntry (struct) | raft.go:2506-2509 | PARTIAL | src/NATS.Server/Raft/CommitQueue.cs:9 | CommitQueue<T> exists as channel wrapper, but no CommittedEntry struct with Index+Entries |
| Entry (struct) | raft.go:2641-2644 | PARTIAL | src/NATS.Server/Raft/RaftWireFormat.cs:73 | RaftEntryWire has Type+Data but is wire-format only; no general Entry type used for proposals |
| RaftState.String() | raft.go:137-149 | PORTED | src/NATS.Server/Raft/RaftStateExtensions.cs:9 | Added Go-style `RaftState.String()` extension mapping to Follower/Leader/Candidate/Closed |
| RaftConfig (struct) | raft.go:301-317 | PORTED | src/NATS.Server/Raft/RaftConfig.cs:8 | Added `RaftConfig` model with Name/Store/Log/Track/Observer/Recovering/ScaleUp fields for parity shape |
| CommittedEntry (struct) | raft.go:2506-2509 | PORTED | src/NATS.Server/Raft/CommitQueue.cs (`CommittedEntry`) | Added explicit committed-entry shape with `Index` and `Entries` list for parity with Go commit delivery payload |
| Entry (struct) | raft.go:2641-2644 | PORTED | src/NATS.Server/Raft/RaftEntry.cs | Added general `RaftEntry` model (`Type`, `Data`) with conversion helpers to/from wire-entry shape |
| EntryType (enum) | raft.go:2605-2619 | PORTED | src/NATS.Server/Raft/RaftWireFormat.cs:54-63 | All types present including EntryCatchup (mapped as RaftEntryType) |
### golang/nats-server/server/raft.go — Exported RaftNode Interface Methods
@@ -104,53 +104,53 @@ Add rows to the Gap Inventory table below. Group by Go source file. Include the
| Go Symbol | Go File:Line | Status | .NET Equivalent | Notes |
|-----------|:-------------|--------|:----------------|-------|
| Propose() | raft.go:909-924 | PARTIAL | src/NATS.Server/Raft/RaftNode.cs:308 | ProposeAsync exists but synchronous replication model, no write-error checking, no proposal queue |
| ProposeMulti() | raft.go:928-945 | MISSING | — | No batch proposal support |
| ProposeMulti() | raft.go:928-945 | PORTED | src/NATS.Server/Raft/RaftNode.cs (`ProposeMultiAsync`) | Added ordered multi-command proposal API returning committed indexes per input command |
| ForwardProposal() | raft.go:949-959 | PARTIAL | src/NATS.Server/Raft/NatsRaftTransport.cs:182 | Transport has ForwardProposal but RaftNode does not call it automatically for non-leaders |
| InstallSnapshot() | raft.go:1295-1311 | PARTIAL | src/NATS.Server/Raft/RaftNode.cs:699 | InstallSnapshotAsync exists but no checkpointing, no WAL compaction, no highwayhash verification |
| CreateSnapshotCheckpoint() | raft.go:1356-1360 | PARTIAL | src/NATS.Server/Raft/RaftNode.cs:519 | CreateSnapshotCheckpointAsync exists but simplified — no async write, no WAL compaction tracking |
| SendSnapshot() | raft.go:1284-1290 | MISSING | — | No direct snapshot send as append entry |
| NeedSnapshot() | raft.go:1551-1555 | MISSING | — | No equivalent check |
| Applied() | raft.go:1183-1185 | MISSING | — | No callback from upper layer for applied index tracking (delegates to Processed) |
| Applied() | raft.go:1183-1185 | PORTED | src/NATS.Server/Raft/RaftNode.cs:403 | Added `Applied(long)` callback returning `(entries,bytes)` and delegating progress to processed tracking |
| Processed() | raft.go:1193-1240 | PARTIAL | src/NATS.Server/Raft/RaftNode.cs:664 | MarkProcessed exists but much simpler — no aflr signaling, no leader state transition, no byte estimation |
| State() | raft.go:2025-2027 | PORTED | src/NATS.Server/Raft/RaftNode.cs:43 | Role property (uses RaftRole enum instead of RaftState) |
| Size() | raft.go:2037-2043 | MISSING | — | No WAL size reporting |
| Progress() | raft.go:2030-2034 | MISSING | — | No combined (index, commit, applied) return |
| Size() | raft.go:2037-2043 | PORTED | src/NATS.Server/Raft/RaftNode.cs:771 | Added `Size()` accessor returning entry count and UTF-8 command-byte estimate from current log contents |
| Progress() | raft.go:2030-2034 | PORTED | src/NATS.Server/Raft/RaftNode.cs:765 | Added `Progress()` accessor returning `(index, commit, applied)` |
| Leader() | raft.go:1712-1717 | PORTED | src/NATS.Server/Raft/RaftNode.cs:42 | IsLeader property |
| LeaderSince() | raft.go:1721-1726 | MISSING | — | No leader-since timestamp tracking |
| LeaderSince() | raft.go:1721-1726 | PORTED | src/NATS.Server/Raft/RaftNode.cs:63 | Added nullable `LeaderSince` timestamp and leadership transition updates |
| Quorum() | raft.go:3070-3083 | PARTIAL | src/NATS.Server/Raft/RaftNode.cs:201 | HasQuorum() exists but uses different window calculation (2x electionTimeout vs Go's lostQuorumInterval) |
| Current() | raft.go:1840-1847 | PARTIAL | src/NATS.Server/Raft/RaftNode.cs:879 | IsCurrent exists but no commit==applied check, no forward-progress polling |
| Healthy() | raft.go:1850-1857 | PARTIAL | src/NATS.Server/Raft/RaftNode.cs:892 | IsHealthy exists but different semantics — checks peer responsiveness, not isCurrent(true) |
| Term() | raft.go:3119-3123 | PORTED | src/NATS.Server/Raft/RaftNode.cs:41 | Term property |
| Leaderless() | raft.go:1876-1883 | MISSING | — | No atomic hasleader flag |
| GroupLeader() | raft.go:1865-1872 | MISSING | — | No leader ID tracking (only IsLeader bool) |
| HadPreviousLeader() | raft.go:1860-1862 | MISSING | — | No pleader atomic flag |
| Leaderless() | raft.go:1876-1883 | PORTED | src/NATS.Server/Raft/RaftNode.cs:65 | Added lock-free `Leaderless` derived from tracked group leader state |
| GroupLeader() | raft.go:1865-1872 | PORTED | src/NATS.Server/Raft/RaftNode.cs:64 | Added `GroupLeader` tracking, updated on election/heartbeat/stepdown |
| HadPreviousLeader() | raft.go:1860-1862 | PORTED | src/NATS.Server/Raft/RaftNode.cs:66 | Added `HadPreviousLeader` flag that remains true after first observed leader |
| StepDown() | raft.go:1900-1977 | PARTIAL | src/NATS.Server/Raft/RaftNode.cs:706 | RequestStepDown exists but no preferred leader selection, no leader transfer, no EntryLeaderTransfer |
| SetObserver() | raft.go:2394-2396 | MISSING | — | No observer mode |
| IsObserver() | raft.go:2387-2391 | MISSING | — | No observer mode |
| SetObserver() | raft.go:2394-2396 | PORTED | src/NATS.Server/Raft/RaftNode.cs:799 | Added `SetObserver(bool)` toggle for observer mode |
| IsObserver() | raft.go:2387-2391 | PORTED | src/NATS.Server/Raft/RaftNode.cs:68 | Added `IsObserver` accessor |
| Campaign() | raft.go:1980-1984 | PARTIAL | src/NATS.Server/Raft/RaftNode.cs:771 | CampaignImmediately exists but no random campaign timeout |
| CampaignImmediately() | raft.go:1987-1993 | PORTED | src/NATS.Server/Raft/RaftNode.cs:771 | CampaignImmediately() |
| ID() | raft.go:2045-2051 | PORTED | src/NATS.Server/Raft/RaftNode.cs:40 | Id property |
| Group() | raft.go:2053-2056 | MISSING | — | No group name tracking |
| Group() | raft.go:2053-2056 | PORTED | src/NATS.Server/Raft/RaftNode.cs:42 | Added `GroupName` tracking on `RaftNode` with constructor support; defaults to node ID when unspecified |
| Peers() | raft.go:2058-2077 | PARTIAL | src/NATS.Server/Raft/RaftNode.cs:872 | GetPeerStates returns Dict but missing Lag calculation |
| ProposeKnownPeers() | raft.go:2080-2089 | MISSING | — | No peer state broadcast |
| UpdateKnownPeers() | raft.go:2092-2096 | MISSING | — | No peer state update |
| ProposeAddPeer() | raft.go:962-983 | PARTIAL | src/NATS.Server/Raft/RaftNode.cs:372 | ProposeAddPeerAsync exists but synchronous replication, no forwarding to leader |
| ProposeRemovePeer() | raft.go:986-1019 | PARTIAL | src/NATS.Server/Raft/RaftNode.cs:415 | ProposeRemovePeerAsync exists but no forwarding, no self-removal handling |
| MembershipChangeInProgress() | raft.go:1021-1025 | PORTED | src/NATS.Server/Raft/RaftNode.cs:67 | MembershipChangeInProgress property |
| AdjustClusterSize() | raft.go:1059-1079 | MISSING | — | No cluster size adjustment |
| AdjustBootClusterSize() | raft.go:1038-1055 | MISSING | — | No boot cluster size adjustment |
| ClusterSize() | raft.go:1029-1033 | MISSING | — | No explicit cluster size property |
| AdjustClusterSize() | raft.go:1059-1079 | PORTED | src/NATS.Server/Raft/RaftNode.cs:790 | Added leader-gated cluster-size adjustment with Go floor behavior (`min 2`) |
| AdjustBootClusterSize() | raft.go:1038-1055 | PORTED | src/NATS.Server/Raft/RaftNode.cs:781 | Added boot-time cluster-size adjustment gated on no current/previous leader |
| ClusterSize() | raft.go:1029-1033 | PORTED | src/NATS.Server/Raft/RaftNode.cs:778 | Added explicit `ClusterSize()` accessor |
| ApplyQ() | raft.go:2106 | PARTIAL | src/NATS.Server/Raft/RaftNode.cs:53 | CommitQueue exists as Channel-based queue, different API than ipQueue |
| PauseApply() | raft.go:1084-1092 | MISSING | — | No apply pausing |
| ResumeApply() | raft.go:1111-1156 | MISSING | — | No apply resuming with replay |
| DrainAndReplaySnapshot() | raft.go:1162-1177 | PARTIAL | src/NATS.Server/Raft/RaftNode.cs:537 | DrainAndReplaySnapshotAsync exists but simplified — no catchup cancellation, no commit preservation |
| LeadChangeC() | raft.go:2110 | MISSING | — | No leader change channel |
| QuitC() | raft.go:2113 | MISSING | — | No quit channel |
| Created() | raft.go:2115-2118 | MISSING | — | No creation timestamp |
| Stop() | raft.go:2120-2122 | MISSING | — | No graceful shutdown (Dispose exists but minimal) |
| WaitForStop() | raft.go:2124-2128 | MISSING | — | No wait-for-stop mechanism |
| Delete() | raft.go:2130-2143 | MISSING | — | No delete with WAL cleanup |
| IsDeleted() | raft.go:2145-2149 | MISSING | — | No deleted flag |
| Created() | raft.go:2115-2118 | PORTED | src/NATS.Server/Raft/RaftNode.cs:43 | Added `CreatedUtc` timestamp captured at node construction and exposed for runtime introspection |
| Stop() | raft.go:2120-2122 | PORTED | src/NATS.Server/Raft/RaftNode.cs:1095 | Added `Stop()` lifecycle API that transitions to follower, clears leader markers, and signals stop waiters |
| WaitForStop() | raft.go:2124-2128 | PORTED | src/NATS.Server/Raft/RaftNode.cs:1104 | Added synchronous `WaitForStop()` backed by stop completion signal |
| Delete() | raft.go:2130-2143 | PORTED | src/NATS.Server/Raft/RaftNode.cs:1109 | Added `Delete()` lifecycle API that stops node, marks deleted, and removes persisted raft directory when configured |
| IsDeleted() | raft.go:2145-2149 | PORTED | src/NATS.Server/Raft/RaftNode.cs:69 | Added `IsDeleted` state accessor |
| RecreateInternalSubs() | raft.go:658-747 | MISSING | — | No NATS internal subscription management |
| IsSystemAccount() | raft.go:648-649 | NOT_APPLICABLE | — | .NET does not have system account NRG routing |
| GetTrafficAccountName() | raft.go:652-656 | NOT_APPLICABLE | — | .NET does not have account NRG routing |
@@ -224,7 +224,7 @@ Add rows to the Gap Inventory table below. Group by Go source file. Include the
| selectNextLeader() | raft.go:1887-1897 | MISSING | — | No next-leader selection by highest index |
| resetElectionTimeout() | raft.go:2241-2243 | PORTED | src/NATS.Server/Raft/RaftNode.cs:737 | ResetElectionTimeout with Timer |
| randElectionTimeout() | raft.go:2235-2238 | PORTED | src/NATS.Server/Raft/RaftNode.cs:727 | RandomizedElectionTimeout |
| randCampaignTimeout() | raft.go:1995-1998 | MISSING | — | No separate campaign timeout |
| randCampaignTimeout() | raft.go:1995-1998 | PORTED | src/NATS.Server/Raft/RaftNode.cs:822 | Added `RandomizedCampaignTimeout()` using Go-equivalent 100-800ms jitter window |
### golang/nats-server/server/raft.go — Wire Format & RPC
@@ -350,12 +350,12 @@ Add rows to the Gap Inventory table below. Group by Go source file. Include the
| Go Symbol | Go File:Line | Status | .NET Equivalent | Notes |
|-----------|:-------------|--------|:----------------|-------|
| Election timeout defaults | raft.go:277-287 | PARTIAL | src/NATS.Server/Raft/RaftNode.cs:56-57 | .NET uses 150-300ms defaults; Go uses 4-9s defaults. Different design choice |
| hbInterval | raft.go:283 | MISSING | — | No heartbeat interval constant |
| lostQuorumInterval | raft.go:284 | MISSING | — | No lost quorum interval |
| observerModeInterval | raft.go:286 | MISSING | — | No observer mode interval |
| peerRemoveTimeout | raft.go:287 | MISSING | — | No peer remove timeout |
| hbInterval | raft.go:283 | PORTED | src/NATS.Server/Raft/RaftNode.cs:10 | Added `HbIntervalDefault = 1s` constant |
| lostQuorumInterval | raft.go:284 | PORTED | src/NATS.Server/Raft/RaftNode.cs:11 | Added `LostQuorumIntervalDefault = 10s` constant |
| observerModeInterval | raft.go:286 | PORTED | src/NATS.Server/Raft/RaftNode.cs:12 | Added `ObserverModeIntervalDefault = 48h` constant |
| peerRemoveTimeout | raft.go:287 | PORTED | src/NATS.Server/Raft/RaftNode.cs:13 | Added `PeerRemoveTimeoutDefault = 5m` constant |
| Error sentinels | raft.go:319-343 | PARTIAL | — | .NET uses InvalidOperationException instead of typed error sentinels |
| noLeader / noVote constants | raft.go:4954-4956 | MISSING | — | No explicit no-leader/no-vote constants |
| noLeader / noVote constants | raft.go:4954-4956 | PORTED | src/NATS.Server/Raft/RaftNode.cs:5 | Added explicit `NoLeader` / `NoVote` empty-string constants |
| paeDropThreshold / paeWarnThreshold | raft.go:4399-4401 | MISSING | — | No pending append entry limits |
| maxBatch / maxEntries | raft.go:3004-3005 | MISSING | — | No proposal batching thresholds |
| extensionState | raft.go:4462-4468 | MISSING | — | No domain extension state tracking |
@@ -387,3 +387,5 @@ After porting work is completed:
|------|--------|----|
| 2026-02-25 | File created with LLM analysis instructions | auto |
| 2026-02-25 | Full gap analysis completed: 196 items inventoried across 12 categories. Summary: 46 PORTED, 38 PARTIAL, 99 MISSING, 13 NOT_APPLICABLE, 0 DEFERRED. Wire format is well-ported; core state machine (run loop, catchup, WAL integration) is largely missing. | claude-opus |
| 2026-02-25 | Ported RAFT API/lifecycle parity batch: LeaderSince/GroupLeader/Leaderless/HadPreviousLeader, observer toggles, Progress/Size/Applied, cluster-size adjustors, stop/delete APIs, campaign timeout jitter, and core timing/leader constants with targeted tests in `RaftNodeParityBatch2Tests`. | codex |
| 2026-02-26 | Ported RAFT parity batch: added `ProposeMultiAsync`, peer parity fields (`Lag`, `Current`) + refresh helpers, explicit `CommittedEntry` payload type, and general `RaftEntry` model/wire conversion helpers with focused tests in `RaftParityBatch3Tests`. | codex |