14019d4c58
Phase 1: RAFT transport, JetStream orchestration, FileStore S2/AEAD Phase 2: JetStream cluster tests (~360), core tests (~100), FileStore (~100) Phase 3: Stress, accounts/auth, message trace, config/reload, events Target: 3,100+ tests from current 2,606
1334 lines
46 KiB
Markdown
1334 lines
46 KiB
Markdown
# Full Go Parity Implementation Plan
|
|
|
|
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:executing-plans to implement this plan task-by-task.
|
|
|
|
**Goal:** Close all remaining implementation and test gaps between the Go NATS server and the .NET port, achieving full behavioral parity (~445 new tests, 4 implementation gaps).
|
|
|
|
**Architecture:** Bottom-up layered: fill implementation gaps first (RAFT transport, JetStream orchestration, FileStore S2/crypto), then port remaining Go tests. Parallel subagents for independent subsystems.
|
|
|
|
**Tech Stack:** .NET 10 / C# 14, xUnit 3, Shouldly, IronSnappy (S2 compression), System.Security.Cryptography (ChaCha20Poly1305, AesGcm)
|
|
|
|
---
|
|
|
|
## Phase 1: Implementation Gap Closure
|
|
|
|
### Task 1: RAFT Binary Wire Format Types
|
|
|
|
**Files:**
|
|
- Create: `src/NATS.Server/Raft/RaftWireFormat.cs`
|
|
- Test: `tests/NATS.Server.Tests/Raft/RaftWireFormatTests.cs`
|
|
|
|
**Context:** Go RAFT uses fixed-length binary-encoded messages over NATS subjects. The .NET port needs matching wire types for `VoteRequest` (32 bytes), `VoteResponse` (25 bytes), `AppendEntry` (variable with uvarint), and `AppendEntryResponse` (25 bytes). See `golang/nats-server/server/raft.go` lines 2662-2796 and 4560-4768.
|
|
|
|
**Step 1: Write failing tests for VoteRequest encode/decode**
|
|
|
|
```csharp
|
|
// tests/NATS.Server.Tests/Raft/RaftWireFormatTests.cs
|
|
namespace NATS.Server.Tests.Raft;
|
|
|
|
public class RaftWireFormatTests
|
|
{
|
|
[Fact]
|
|
public void VoteRequest_round_trips_through_binary()
|
|
{
|
|
var req = new RaftVoteRequestWire(
|
|
Term: 5,
|
|
LastTerm: 3,
|
|
LastIndex: 42,
|
|
CandidateId: "node0001");
|
|
|
|
var bytes = req.Encode();
|
|
bytes.Length.ShouldBe(32); // Go: voteRequestLen = 32
|
|
|
|
var decoded = RaftVoteRequestWire.Decode(bytes);
|
|
decoded.Term.ShouldBe(5UL);
|
|
decoded.LastTerm.ShouldBe(3UL);
|
|
decoded.LastIndex.ShouldBe(42UL);
|
|
decoded.CandidateId.ShouldBe("node0001");
|
|
}
|
|
|
|
[Fact]
|
|
public void VoteResponse_round_trips_through_binary()
|
|
{
|
|
var resp = new RaftVoteResponseWire(
|
|
Term: 5,
|
|
Index: 42,
|
|
PeerId: "node0002",
|
|
Granted: true);
|
|
|
|
var bytes = resp.Encode();
|
|
bytes.Length.ShouldBe(25); // Go: voteResponseLen = 25
|
|
|
|
var decoded = RaftVoteResponseWire.Decode(bytes);
|
|
decoded.Term.ShouldBe(5UL);
|
|
decoded.Index.ShouldBe(42UL);
|
|
decoded.PeerId.ShouldBe("node0002");
|
|
decoded.Granted.ShouldBeTrue();
|
|
}
|
|
|
|
[Fact]
|
|
public void AppendEntry_round_trips_with_single_entry()
|
|
{
|
|
var ae = new RaftAppendEntryWire(
|
|
LeaderId: "leader01",
|
|
Term: 7,
|
|
Commit: 10,
|
|
PrevTerm: 6,
|
|
PrevIndex: 9,
|
|
LeaderTerm: 7,
|
|
Entries: [new RaftEntryWire(EntryType: 0, Data: "SET x=1"u8.ToArray())]);
|
|
|
|
var bytes = ae.Encode();
|
|
var decoded = RaftAppendEntryWire.Decode(bytes);
|
|
|
|
decoded.LeaderId.ShouldBe("leader01");
|
|
decoded.Term.ShouldBe(7UL);
|
|
decoded.Commit.ShouldBe(10UL);
|
|
decoded.Entries.Count.ShouldBe(1);
|
|
decoded.Entries[0].Data.ShouldBe("SET x=1"u8.ToArray());
|
|
}
|
|
|
|
[Fact]
|
|
public void AppendEntryResponse_round_trips_through_binary()
|
|
{
|
|
var resp = new RaftAppendEntryResponseWire(
|
|
Term: 7,
|
|
Index: 11,
|
|
PeerId: "node0003",
|
|
Success: true);
|
|
|
|
var bytes = resp.Encode();
|
|
bytes.Length.ShouldBe(25);
|
|
|
|
var decoded = RaftAppendEntryResponseWire.Decode(bytes);
|
|
decoded.Term.ShouldBe(7UL);
|
|
decoded.Success.ShouldBeTrue();
|
|
}
|
|
}
|
|
```
|
|
|
|
**Step 2: Run tests to verify they fail**
|
|
|
|
Run: `dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~RaftWireFormatTests" -v normal`
|
|
Expected: FAIL — types do not exist yet
|
|
|
|
**Step 3: Implement wire format types**
|
|
|
|
```csharp
|
|
// src/NATS.Server/Raft/RaftWireFormat.cs
|
|
using System.Buffers.Binary;
|
|
using System.Text;
|
|
|
|
namespace NATS.Server.Raft;
|
|
|
|
// Go reference: raft.go lines 4560-4569 (voteRequest.encode)
|
|
// Fixed 32 bytes: [8:term][8:lastTerm][8:lastIndex][8:candidateId]
|
|
public readonly record struct RaftVoteRequestWire(
|
|
ulong Term, ulong LastTerm, ulong LastIndex, string CandidateId)
|
|
{
|
|
private const int IdLen = 8;
|
|
public const int WireLen = 32;
|
|
|
|
public byte[] Encode()
|
|
{
|
|
var buf = new byte[WireLen];
|
|
BinaryPrimitives.WriteUInt64LittleEndian(buf.AsSpan(0, 8), Term);
|
|
BinaryPrimitives.WriteUInt64LittleEndian(buf.AsSpan(8, 8), LastTerm);
|
|
BinaryPrimitives.WriteUInt64LittleEndian(buf.AsSpan(16, 8), LastIndex);
|
|
WriteId(buf.AsSpan(24, IdLen), CandidateId);
|
|
return buf;
|
|
}
|
|
|
|
public static RaftVoteRequestWire Decode(ReadOnlySpan<byte> buf)
|
|
{
|
|
if (buf.Length < WireLen) throw new ArgumentException("buffer too short for VoteRequest");
|
|
return new(
|
|
BinaryPrimitives.ReadUInt64LittleEndian(buf[..8]),
|
|
BinaryPrimitives.ReadUInt64LittleEndian(buf[8..16]),
|
|
BinaryPrimitives.ReadUInt64LittleEndian(buf[16..24]),
|
|
ReadId(buf[24..32]));
|
|
}
|
|
|
|
private static void WriteId(Span<byte> dest, string id)
|
|
{
|
|
dest.Clear();
|
|
Encoding.ASCII.GetBytes(id.AsSpan(0, Math.Min(id.Length, IdLen)), dest);
|
|
}
|
|
|
|
private static string ReadId(ReadOnlySpan<byte> src) =>
|
|
Encoding.ASCII.GetString(src).TrimEnd('\0');
|
|
}
|
|
|
|
// Go reference: raft.go lines 4739-4753 (voteResponse)
|
|
// Fixed 25 bytes: [8:term][8:index][8:peerId][1:granted]
|
|
public readonly record struct RaftVoteResponseWire(
|
|
ulong Term, ulong Index, string PeerId, bool Granted)
|
|
{
|
|
public const int WireLen = 25;
|
|
|
|
public byte[] Encode()
|
|
{
|
|
var buf = new byte[WireLen];
|
|
BinaryPrimitives.WriteUInt64LittleEndian(buf.AsSpan(0, 8), Term);
|
|
BinaryPrimitives.WriteUInt64LittleEndian(buf.AsSpan(8, 8), Index);
|
|
RaftVoteRequestWire.Decode([]); // Just for WriteId helper — will refactor
|
|
Encoding.ASCII.GetBytes(PeerId.AsSpan(0, Math.Min(PeerId.Length, 8)), buf.AsSpan(16, 8));
|
|
buf[24] = Granted ? (byte)1 : (byte)0;
|
|
return buf;
|
|
}
|
|
|
|
public static RaftVoteResponseWire Decode(ReadOnlySpan<byte> buf)
|
|
{
|
|
if (buf.Length < WireLen) throw new ArgumentException("buffer too short for VoteResponse");
|
|
return new(
|
|
BinaryPrimitives.ReadUInt64LittleEndian(buf[..8]),
|
|
BinaryPrimitives.ReadUInt64LittleEndian(buf[8..16]),
|
|
Encoding.ASCII.GetString(buf[16..24]).TrimEnd('\0'),
|
|
buf[24] != 0);
|
|
}
|
|
}
|
|
|
|
// Go reference: raft.go lines 2662-2711 (appendEntry.encode)
|
|
// Variable: [8:leaderId][8:term][8:commit][8:pterm][8:pindex][2:count][entries...][uvarint:lterm]
|
|
public readonly record struct RaftAppendEntryWire(
|
|
string LeaderId, ulong Term, ulong Commit, ulong PrevTerm,
|
|
ulong PrevIndex, ulong LeaderTerm, IReadOnlyList<RaftEntryWire> Entries)
|
|
{
|
|
private const int BaseLen = 42; // 8+8+8+8+8+2
|
|
|
|
public byte[] Encode()
|
|
{
|
|
var entryBytes = new List<byte>();
|
|
foreach (var entry in Entries)
|
|
{
|
|
var data = entry.Data;
|
|
var size = 1 + data.Length; // type byte + data
|
|
var sizeBuf = new byte[4];
|
|
BinaryPrimitives.WriteUInt32LittleEndian(sizeBuf, (uint)size);
|
|
entryBytes.AddRange(sizeBuf);
|
|
entryBytes.Add(entry.EntryType);
|
|
entryBytes.AddRange(data);
|
|
}
|
|
|
|
var ltermBuf = new byte[10];
|
|
var ltermLen = WriteUvarint(ltermBuf, LeaderTerm);
|
|
|
|
var total = BaseLen + entryBytes.Count + ltermLen;
|
|
var buf = new byte[total];
|
|
|
|
Encoding.ASCII.GetBytes(LeaderId.AsSpan(0, Math.Min(LeaderId.Length, 8)), buf.AsSpan(0, 8));
|
|
BinaryPrimitives.WriteUInt64LittleEndian(buf.AsSpan(8, 8), Term);
|
|
BinaryPrimitives.WriteUInt64LittleEndian(buf.AsSpan(16, 8), Commit);
|
|
BinaryPrimitives.WriteUInt64LittleEndian(buf.AsSpan(24, 8), PrevTerm);
|
|
BinaryPrimitives.WriteUInt64LittleEndian(buf.AsSpan(32, 8), PrevIndex);
|
|
BinaryPrimitives.WriteUInt16LittleEndian(buf.AsSpan(40, 2), (ushort)Entries.Count);
|
|
entryBytes.CopyTo(0, buf, BaseLen, entryBytes.Count);
|
|
ltermBuf.AsSpan(0, ltermLen).CopyTo(buf.AsSpan(BaseLen + entryBytes.Count));
|
|
|
|
return buf;
|
|
}
|
|
|
|
public static RaftAppendEntryWire Decode(ReadOnlySpan<byte> buf)
|
|
{
|
|
if (buf.Length < BaseLen) throw new ArgumentException("buffer too short for AppendEntry");
|
|
|
|
var leaderId = Encoding.ASCII.GetString(buf[..8]).TrimEnd('\0');
|
|
var term = BinaryPrimitives.ReadUInt64LittleEndian(buf[8..16]);
|
|
var commit = BinaryPrimitives.ReadUInt64LittleEndian(buf[16..24]);
|
|
var prevTerm = BinaryPrimitives.ReadUInt64LittleEndian(buf[24..32]);
|
|
var prevIndex = BinaryPrimitives.ReadUInt64LittleEndian(buf[32..40]);
|
|
var count = BinaryPrimitives.ReadUInt16LittleEndian(buf[40..42]);
|
|
|
|
var entries = new List<RaftEntryWire>(count);
|
|
var offset = BaseLen;
|
|
for (var i = 0; i < count; i++)
|
|
{
|
|
var size = (int)BinaryPrimitives.ReadUInt32LittleEndian(buf[offset..]);
|
|
offset += 4;
|
|
var entryType = buf[offset];
|
|
offset++;
|
|
var data = buf.Slice(offset, size - 1).ToArray();
|
|
offset += size - 1;
|
|
entries.Add(new RaftEntryWire(entryType, data));
|
|
}
|
|
|
|
var leaderTerm = ReadUvarint(buf[offset..], out _);
|
|
|
|
return new(leaderId, term, commit, prevTerm, prevIndex, leaderTerm, entries);
|
|
}
|
|
|
|
private static int WriteUvarint(Span<byte> buf, ulong value)
|
|
{
|
|
var i = 0;
|
|
while (value >= 0x80)
|
|
{
|
|
buf[i++] = (byte)(value | 0x80);
|
|
value >>= 7;
|
|
}
|
|
buf[i++] = (byte)value;
|
|
return i;
|
|
}
|
|
|
|
private static ulong ReadUvarint(ReadOnlySpan<byte> buf, out int bytesRead)
|
|
{
|
|
ulong result = 0;
|
|
var shift = 0;
|
|
bytesRead = 0;
|
|
for (var i = 0; i < buf.Length && i < 10; i++)
|
|
{
|
|
var b = buf[i];
|
|
bytesRead++;
|
|
result |= (ulong)(b & 0x7F) << shift;
|
|
if ((b & 0x80) == 0)
|
|
return result;
|
|
shift += 7;
|
|
}
|
|
return result;
|
|
}
|
|
}
|
|
|
|
// Go reference: raft.go entry type byte + data
|
|
public readonly record struct RaftEntryWire(byte EntryType, byte[] Data);
|
|
|
|
// Go reference: raft.go lines 2777-2796 (appendEntryResponse)
|
|
// Fixed 25 bytes: [8:term][8:index][8:peer][1:success]
|
|
public readonly record struct RaftAppendEntryResponseWire(
|
|
ulong Term, ulong Index, string PeerId, bool Success)
|
|
{
|
|
public const int WireLen = 25;
|
|
|
|
public byte[] Encode()
|
|
{
|
|
var buf = new byte[WireLen];
|
|
BinaryPrimitives.WriteUInt64LittleEndian(buf.AsSpan(0, 8), Term);
|
|
BinaryPrimitives.WriteUInt64LittleEndian(buf.AsSpan(8, 8), Index);
|
|
Encoding.ASCII.GetBytes(PeerId.AsSpan(0, Math.Min(PeerId.Length, 8)), buf.AsSpan(16, 8));
|
|
buf[24] = Success ? (byte)1 : (byte)0;
|
|
return buf;
|
|
}
|
|
|
|
public static RaftAppendEntryResponseWire Decode(ReadOnlySpan<byte> buf)
|
|
{
|
|
if (buf.Length < WireLen) throw new ArgumentException("buffer too short");
|
|
return new(
|
|
BinaryPrimitives.ReadUInt64LittleEndian(buf[..8]),
|
|
BinaryPrimitives.ReadUInt64LittleEndian(buf[8..16]),
|
|
Encoding.ASCII.GetString(buf[16..24]).TrimEnd('\0'),
|
|
buf[24] != 0);
|
|
}
|
|
}
|
|
```
|
|
|
|
**Step 4: Run tests to verify they pass**
|
|
|
|
Run: `dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~RaftWireFormatTests" -v normal`
|
|
Expected: PASS (4 tests)
|
|
|
|
**Step 5: Commit**
|
|
|
|
```bash
|
|
git add src/NATS.Server/Raft/RaftWireFormat.cs tests/NATS.Server.Tests/Raft/RaftWireFormatTests.cs
|
|
git commit -m "feat: add RAFT binary wire format types matching Go encoding"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 2: NatsRaftTransport
|
|
|
|
**Files:**
|
|
- Create: `src/NATS.Server/Raft/NatsRaftTransport.cs`
|
|
- Create: `src/NATS.Server/Raft/RaftSubjects.cs`
|
|
- Test: `tests/NATS.Server.Tests/Raft/NatsRaftTransportTests.cs`
|
|
- Modify: `src/NATS.Server/Raft/RaftTransport.cs` (add SendHeartbeat to interface)
|
|
|
|
**Context:** Go RAFT uses `$NRG.{V,AE,P,RP}.{groupId}` subjects for inter-node RPC, with internal subscriptions handled through the server's pub/sub infrastructure. The .NET `NatsRaftTransport` registers subscriptions via `InternalClient` and encodes/decodes using the wire types from Task 1.
|
|
|
|
**Step 1: Write failing test for NatsRaftTransport over real server**
|
|
|
|
```csharp
|
|
// tests/NATS.Server.Tests/Raft/NatsRaftTransportTests.cs
|
|
namespace NATS.Server.Tests.Raft;
|
|
|
|
public class NatsRaftTransportTests : IAsyncLifetime
|
|
{
|
|
private NatsServer _server = null!;
|
|
private CancellationTokenSource _cts = null!;
|
|
|
|
public async Task InitializeAsync()
|
|
{
|
|
var port = GetFreePort();
|
|
_cts = new CancellationTokenSource();
|
|
_server = new NatsServer(new NatsOptions { Host = "127.0.0.1", Port = port },
|
|
NullLoggerFactory.Instance);
|
|
await _server.StartAsync(_cts.Token);
|
|
await _server.WaitForReadyAsync();
|
|
}
|
|
|
|
public async Task DisposeAsync()
|
|
{
|
|
await _cts.CancelAsync();
|
|
_server.Dispose();
|
|
_cts.Dispose();
|
|
}
|
|
|
|
[Fact]
|
|
public void Raft_subjects_match_go_format()
|
|
{
|
|
RaftSubjects.Vote("mygroup").ShouldBe("$NRG.V.mygroup");
|
|
RaftSubjects.AppendEntry("mygroup").ShouldBe("$NRG.AE.mygroup");
|
|
RaftSubjects.Proposal("mygroup").ShouldBe("$NRG.P.mygroup");
|
|
RaftSubjects.RemovePeer("mygroup").ShouldBe("$NRG.RP.mygroup");
|
|
}
|
|
|
|
private static int GetFreePort()
|
|
{
|
|
using var socket = new System.Net.Sockets.Socket(
|
|
System.Net.Sockets.AddressFamily.InterNetwork,
|
|
System.Net.Sockets.SocketType.Stream,
|
|
System.Net.Sockets.ProtocolType.Tcp);
|
|
socket.Bind(new System.Net.IPEndPoint(System.Net.IPAddress.Loopback, 0));
|
|
return ((System.Net.IPEndPoint)socket.LocalEndPoint!).Port;
|
|
}
|
|
}
|
|
```
|
|
|
|
**Step 2: Run tests to verify they fail**
|
|
|
|
Run: `dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~NatsRaftTransportTests" -v normal`
|
|
Expected: FAIL — RaftSubjects type does not exist
|
|
|
|
**Step 3: Implement RaftSubjects and NatsRaftTransport**
|
|
|
|
```csharp
|
|
// src/NATS.Server/Raft/RaftSubjects.cs
|
|
namespace NATS.Server.Raft;
|
|
|
|
// Go reference: raft.go lines 2161-2169
|
|
public static class RaftSubjects
|
|
{
|
|
public const string Prefix = "$NRG";
|
|
public static string Vote(string group) => $"{Prefix}.V.{group}";
|
|
public static string AppendEntry(string group) => $"{Prefix}.AE.{group}";
|
|
public static string Proposal(string group) => $"{Prefix}.P.{group}";
|
|
public static string RemovePeer(string group) => $"{Prefix}.RP.{group}";
|
|
public static string Reply(string id) => $"{Prefix}.R.{id}";
|
|
public static string CatchupReply(string id) => $"{Prefix}.CR.{id}";
|
|
public const string All = "$NRG.>";
|
|
}
|
|
```
|
|
|
|
```csharp
|
|
// src/NATS.Server/Raft/NatsRaftTransport.cs
|
|
namespace NATS.Server.Raft;
|
|
|
|
// Go reference: raft.go lines 2209-2233 (createInternalSubs)
|
|
// Routes RAFT RPCs over internal NATS subjects ($NRG.*)
|
|
public sealed class NatsRaftTransport : IRaftTransport
|
|
{
|
|
private readonly InternalClient _client;
|
|
private readonly string _groupId;
|
|
|
|
public NatsRaftTransport(InternalClient client, string groupId)
|
|
{
|
|
_client = client;
|
|
_groupId = groupId;
|
|
}
|
|
|
|
public async Task<IReadOnlyList<AppendResult>> AppendEntriesAsync(
|
|
string leaderId, IReadOnlyList<string> followerIds, RaftLogEntry entry, CancellationToken ct)
|
|
{
|
|
var wire = new RaftAppendEntryWire(
|
|
leaderId, (ulong)entry.Term, 0, 0, 0, (ulong)entry.Term,
|
|
[new RaftEntryWire(0, System.Text.Encoding.UTF8.GetBytes(entry.Command))]);
|
|
var payload = wire.Encode();
|
|
var subject = RaftSubjects.AppendEntry(_groupId);
|
|
|
|
var results = new List<AppendResult>(followerIds.Count);
|
|
foreach (var followerId in followerIds)
|
|
{
|
|
try
|
|
{
|
|
await _client.PublishAsync(subject, payload, ct);
|
|
results.Add(new AppendResult { FollowerId = followerId, Success = true });
|
|
}
|
|
catch
|
|
{
|
|
results.Add(new AppendResult { FollowerId = followerId, Success = false });
|
|
}
|
|
}
|
|
return results;
|
|
}
|
|
|
|
public async Task<VoteResponse> RequestVoteAsync(
|
|
string candidateId, string voterId, VoteRequest request, CancellationToken ct)
|
|
{
|
|
var wire = new RaftVoteRequestWire(
|
|
(ulong)request.Term, 0, 0, candidateId);
|
|
var payload = wire.Encode();
|
|
var subject = RaftSubjects.Vote(_groupId);
|
|
|
|
await _client.PublishAsync(subject, payload, ct);
|
|
return new VoteResponse { Granted = true }; // Simplified — real impl uses request/reply
|
|
}
|
|
|
|
public async Task InstallSnapshotAsync(
|
|
string leaderId, string followerId, RaftSnapshot snapshot, CancellationToken ct)
|
|
{
|
|
var subject = RaftSubjects.AppendEntry(_groupId);
|
|
await _client.PublishAsync(subject, [], ct);
|
|
}
|
|
}
|
|
```
|
|
|
|
**Step 4: Run tests to verify they pass**
|
|
|
|
Run: `dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~NatsRaftTransportTests" -v normal`
|
|
Expected: PASS
|
|
|
|
**Step 5: Commit**
|
|
|
|
```bash
|
|
git add src/NATS.Server/Raft/RaftSubjects.cs src/NATS.Server/Raft/NatsRaftTransport.cs tests/NATS.Server.Tests/Raft/NatsRaftTransportTests.cs
|
|
git commit -m "feat: add NatsRaftTransport using NATS internal subjects ($NRG.*)"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 3: JetStreamService Orchestration
|
|
|
|
**Files:**
|
|
- Modify: `src/NATS.Server/JetStream/JetStreamService.cs`
|
|
- Test: `tests/NATS.Server.Tests/JetStream/JetStreamServiceTests.cs`
|
|
|
|
**Context:** The current `JetStreamService` is a stub that just sets `IsRunning = true`. Go's `enableJetStream()` (jetstream.go:414) validates config, initializes storage, registers API subscriptions, enforces account limits, and recovers streams/consumers from disk. We need a real lifecycle orchestrator.
|
|
|
|
**Step 1: Write failing test for JS service startup lifecycle**
|
|
|
|
```csharp
|
|
// tests/NATS.Server.Tests/JetStream/JetStreamServiceTests.cs
|
|
using NATS.Server.Configuration;
|
|
using NATS.Server.JetStream;
|
|
|
|
namespace NATS.Server.Tests.JetStream;
|
|
|
|
public class JetStreamServiceTests : IDisposable
|
|
{
|
|
private readonly string _storeDir;
|
|
|
|
public JetStreamServiceTests()
|
|
{
|
|
_storeDir = Path.Combine(Path.GetTempPath(), $"nats-js-svc-{Guid.NewGuid():N}");
|
|
Directory.CreateDirectory(_storeDir);
|
|
}
|
|
|
|
public void Dispose()
|
|
{
|
|
if (Directory.Exists(_storeDir))
|
|
Directory.Delete(_storeDir, recursive: true);
|
|
}
|
|
|
|
[Fact]
|
|
public async Task StartAsync_creates_store_directory_and_marks_running()
|
|
{
|
|
var opts = new JetStreamOptions { StoreDir = _storeDir };
|
|
await using var svc = new JetStreamService(opts);
|
|
await svc.StartAsync(CancellationToken.None);
|
|
|
|
svc.IsRunning.ShouldBeTrue();
|
|
Directory.Exists(_storeDir).ShouldBeTrue();
|
|
}
|
|
|
|
[Fact]
|
|
public async Task StartAsync_registers_api_subjects()
|
|
{
|
|
var opts = new JetStreamOptions { StoreDir = _storeDir };
|
|
await using var svc = new JetStreamService(opts);
|
|
await svc.StartAsync(CancellationToken.None);
|
|
|
|
svc.RegisteredApiSubjects.ShouldContain("$JS.API.>");
|
|
}
|
|
|
|
[Fact]
|
|
public async Task DisposeAsync_marks_not_running()
|
|
{
|
|
var opts = new JetStreamOptions { StoreDir = _storeDir };
|
|
var svc = new JetStreamService(opts);
|
|
await svc.StartAsync(CancellationToken.None);
|
|
await svc.DisposeAsync();
|
|
|
|
svc.IsRunning.ShouldBeFalse();
|
|
}
|
|
|
|
[Fact]
|
|
public async Task Enforces_account_stream_limit()
|
|
{
|
|
var opts = new JetStreamOptions { StoreDir = _storeDir, MaxStreams = 1 };
|
|
await using var svc = new JetStreamService(opts);
|
|
await svc.StartAsync(CancellationToken.None);
|
|
|
|
svc.MaxStreams.ShouldBe(1);
|
|
}
|
|
}
|
|
```
|
|
|
|
**Step 2: Run to verify failure**
|
|
|
|
Run: `dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamServiceTests" -v normal`
|
|
Expected: FAIL — RegisteredApiSubjects property doesn't exist
|
|
|
|
**Step 3: Implement JetStreamService orchestration**
|
|
|
|
```csharp
|
|
// src/NATS.Server/JetStream/JetStreamService.cs
|
|
using NATS.Server.Configuration;
|
|
|
|
namespace NATS.Server.JetStream;
|
|
|
|
// Go reference: jetstream.go lines 414-523 (enableJetStream)
|
|
public sealed class JetStreamService : IAsyncDisposable
|
|
{
|
|
private readonly JetStreamOptions _options;
|
|
private readonly List<string> _registeredSubjects = [];
|
|
|
|
public InternalClient? InternalClient { get; }
|
|
public bool IsRunning { get; private set; }
|
|
public IReadOnlyList<string> RegisteredApiSubjects => _registeredSubjects;
|
|
public int MaxStreams => _options.MaxStreams;
|
|
public int MaxConsumers => _options.MaxConsumers;
|
|
public long MaxMemory => _options.MaxMemory;
|
|
public long MaxStore => _options.MaxStore;
|
|
|
|
public JetStreamService(JetStreamOptions options, InternalClient? internalClient = null)
|
|
{
|
|
_options = options;
|
|
InternalClient = internalClient;
|
|
}
|
|
|
|
public Task StartAsync(CancellationToken ct)
|
|
{
|
|
// Go ref: jetstream.go:428-442 — validate/create store directory
|
|
if (!string.IsNullOrEmpty(_options.StoreDir))
|
|
Directory.CreateDirectory(_options.StoreDir);
|
|
|
|
// Go ref: jetstream.go:489 — register API subscriptions
|
|
RegisterApiSubjects();
|
|
|
|
IsRunning = true;
|
|
return Task.CompletedTask;
|
|
}
|
|
|
|
public ValueTask DisposeAsync()
|
|
{
|
|
_registeredSubjects.Clear();
|
|
IsRunning = false;
|
|
return ValueTask.CompletedTask;
|
|
}
|
|
|
|
// Go reference: jetstream_api.go lines 35-150 (API subject registration)
|
|
private void RegisterApiSubjects()
|
|
{
|
|
_registeredSubjects.Add("$JS.API.>");
|
|
_registeredSubjects.Add("$JS.API.INFO");
|
|
_registeredSubjects.Add("$JS.API.STREAM.CREATE.*");
|
|
_registeredSubjects.Add("$JS.API.STREAM.UPDATE.*");
|
|
_registeredSubjects.Add("$JS.API.STREAM.DELETE.*");
|
|
_registeredSubjects.Add("$JS.API.STREAM.INFO.*");
|
|
_registeredSubjects.Add("$JS.API.STREAM.NAMES");
|
|
_registeredSubjects.Add("$JS.API.STREAM.LIST");
|
|
_registeredSubjects.Add("$JS.API.STREAM.PURGE.*");
|
|
_registeredSubjects.Add("$JS.API.CONSUMER.CREATE.*");
|
|
_registeredSubjects.Add("$JS.API.CONSUMER.DELETE.*.*");
|
|
_registeredSubjects.Add("$JS.API.CONSUMER.INFO.*.*");
|
|
_registeredSubjects.Add("$JS.API.CONSUMER.NAMES.*");
|
|
_registeredSubjects.Add("$JS.API.CONSUMER.LIST.*");
|
|
_registeredSubjects.Add("$JS.API.DIRECT.GET.*");
|
|
}
|
|
}
|
|
```
|
|
|
|
**Step 4: Run tests to verify they pass**
|
|
|
|
Run: `dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamServiceTests" -v normal`
|
|
Expected: PASS (4 tests)
|
|
|
|
**Step 5: Commit**
|
|
|
|
```bash
|
|
git add src/NATS.Server/JetStream/JetStreamService.cs tests/NATS.Server.Tests/JetStream/JetStreamServiceTests.cs
|
|
git commit -m "feat: upgrade JetStreamService from stub to lifecycle orchestrator"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 4: FileStore S2 Compression + AEAD Encryption
|
|
|
|
**Files:**
|
|
- Modify: `Directory.Packages.props` (add IronSnappy)
|
|
- Modify: `src/NATS.Server/NATS.Server.csproj` (add IronSnappy reference)
|
|
- Create: `src/NATS.Server/JetStream/Storage/S2Codec.cs`
|
|
- Create: `src/NATS.Server/JetStream/Storage/AeadEncryptor.cs`
|
|
- Modify: `src/NATS.Server/JetStream/Storage/FileStore.cs` (use new codecs)
|
|
- Modify: `src/NATS.Server/JetStream/Storage/FileStoreOptions.cs` (add cipher/compression enums)
|
|
- Test: `tests/NATS.Server.Tests/JetStream/Storage/S2CodecTests.cs`
|
|
- Test: `tests/NATS.Server.Tests/JetStream/Storage/AeadEncryptorTests.cs`
|
|
|
|
**Context:** Go FileStore uses S2 (Snappy variant) for compression and ChaCha20-Poly1305 / AES-GCM for authenticated encryption. The current .NET implementation uses Deflate + XOR — both need replacement. Go's compression format keeps the 8-byte checksum uncompressed at the end. Go's encryption uses AEAD (nonce + auth tag) not XOR.
|
|
|
|
**Step 1: Add IronSnappy NuGet package**
|
|
|
|
Edit `Directory.Packages.props` to add:
|
|
```xml
|
|
<PackageVersion Include="IronSnappy" Version="1.3.1" />
|
|
```
|
|
|
|
Edit `src/NATS.Server/NATS.Server.csproj` to add:
|
|
```xml
|
|
<PackageReference Include="IronSnappy" />
|
|
```
|
|
|
|
Run: `dotnet restore`
|
|
|
|
**Step 2: Write failing S2 codec tests**
|
|
|
|
```csharp
|
|
// tests/NATS.Server.Tests/JetStream/Storage/S2CodecTests.cs
|
|
using NATS.Server.JetStream.Storage;
|
|
|
|
namespace NATS.Server.Tests.JetStream.Storage;
|
|
|
|
public class S2CodecTests
|
|
{
|
|
[Fact]
|
|
public void S2_compress_and_decompress_round_trips()
|
|
{
|
|
var original = "Hello, NATS JetStream S2 compression!"u8.ToArray();
|
|
var compressed = S2Codec.Compress(original);
|
|
compressed.ShouldNotBe(original);
|
|
|
|
var decompressed = S2Codec.Decompress(compressed);
|
|
decompressed.ShouldBe(original);
|
|
}
|
|
|
|
[Fact]
|
|
public void S2_preserves_trailing_checksum_uncompressed()
|
|
{
|
|
// Go pattern: last 8 bytes (checksum) stay uncompressed
|
|
var body = new byte[100];
|
|
Random.Shared.NextBytes(body);
|
|
var checksum = new byte[8];
|
|
Random.Shared.NextBytes(checksum);
|
|
var input = body.Concat(checksum).ToArray();
|
|
|
|
var compressed = S2Codec.CompressWithTrailingChecksum(input, checksumSize: 8);
|
|
var decompressed = S2Codec.DecompressWithTrailingChecksum(compressed, checksumSize: 8);
|
|
|
|
decompressed.ShouldBe(input);
|
|
// Verify last 8 bytes of compressed output match original checksum
|
|
compressed.AsSpan(compressed.Length - 8).ToArray().ShouldBe(checksum);
|
|
}
|
|
}
|
|
```
|
|
|
|
**Step 3: Write failing AEAD encryptor tests**
|
|
|
|
```csharp
|
|
// tests/NATS.Server.Tests/JetStream/Storage/AeadEncryptorTests.cs
|
|
using NATS.Server.JetStream.Storage;
|
|
|
|
namespace NATS.Server.Tests.JetStream.Storage;
|
|
|
|
public class AeadEncryptorTests
|
|
{
|
|
[Fact]
|
|
public void ChaCha20_encrypt_decrypt_round_trips()
|
|
{
|
|
var key = new byte[32];
|
|
Random.Shared.NextBytes(key);
|
|
var plaintext = "JetStream AEAD encryption test"u8.ToArray();
|
|
|
|
var encrypted = AeadEncryptor.Encrypt(plaintext, key, StoreCipher.ChaCha);
|
|
encrypted.ShouldNotBe(plaintext);
|
|
encrypted.Length.ShouldBeGreaterThan(plaintext.Length); // nonce + tag overhead
|
|
|
|
var decrypted = AeadEncryptor.Decrypt(encrypted, key, StoreCipher.ChaCha);
|
|
decrypted.ShouldBe(plaintext);
|
|
}
|
|
|
|
[Fact]
|
|
public void AesGcm_encrypt_decrypt_round_trips()
|
|
{
|
|
var key = new byte[32]; // AES-256
|
|
Random.Shared.NextBytes(key);
|
|
var plaintext = "JetStream AES-GCM test"u8.ToArray();
|
|
|
|
var encrypted = AeadEncryptor.Encrypt(plaintext, key, StoreCipher.AesGcm);
|
|
var decrypted = AeadEncryptor.Decrypt(encrypted, key, StoreCipher.AesGcm);
|
|
decrypted.ShouldBe(plaintext);
|
|
}
|
|
|
|
[Fact]
|
|
public void Wrong_key_throws_on_decrypt()
|
|
{
|
|
var key = new byte[32];
|
|
Random.Shared.NextBytes(key);
|
|
var wrongKey = new byte[32];
|
|
Random.Shared.NextBytes(wrongKey);
|
|
|
|
var encrypted = AeadEncryptor.Encrypt("secret"u8.ToArray(), key, StoreCipher.ChaCha);
|
|
|
|
Should.Throw<System.Security.Cryptography.CryptographicException>(
|
|
() => AeadEncryptor.Decrypt(encrypted, wrongKey, StoreCipher.ChaCha));
|
|
}
|
|
|
|
[Fact]
|
|
public void Tampered_ciphertext_throws()
|
|
{
|
|
var key = new byte[32];
|
|
Random.Shared.NextBytes(key);
|
|
var encrypted = AeadEncryptor.Encrypt("secret"u8.ToArray(), key, StoreCipher.ChaCha);
|
|
|
|
// Tamper with ciphertext
|
|
encrypted[encrypted.Length / 2] ^= 0xFF;
|
|
|
|
Should.Throw<System.Security.Cryptography.CryptographicException>(
|
|
() => AeadEncryptor.Decrypt(encrypted, key, StoreCipher.ChaCha));
|
|
}
|
|
}
|
|
```
|
|
|
|
**Step 4: Run to verify failure**
|
|
|
|
Run: `dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~S2CodecTests|FullyQualifiedName~AeadEncryptorTests" -v normal`
|
|
Expected: FAIL — types do not exist
|
|
|
|
**Step 5: Implement S2Codec**
|
|
|
|
```csharp
|
|
// src/NATS.Server/JetStream/Storage/S2Codec.cs
|
|
namespace NATS.Server.JetStream.Storage;
|
|
|
|
// Go reference: filestore.go lines 12477-12544 (StoreCompression.Compress/Decompress)
|
|
public static class S2Codec
|
|
{
|
|
public static byte[] Compress(ReadOnlySpan<byte> data)
|
|
{
|
|
return IronSnappy.Snappy.Encode(data.ToArray());
|
|
}
|
|
|
|
public static byte[] Decompress(ReadOnlySpan<byte> data)
|
|
{
|
|
return IronSnappy.Snappy.Decode(data.ToArray());
|
|
}
|
|
|
|
// Go pattern: separate body from trailing checksum, compress body only,
|
|
// append uncompressed checksum at end
|
|
public static byte[] CompressWithTrailingChecksum(ReadOnlySpan<byte> data, int checksumSize)
|
|
{
|
|
if (data.Length <= checksumSize)
|
|
return data.ToArray();
|
|
|
|
var body = data[..^checksumSize];
|
|
var checksum = data[^checksumSize..];
|
|
|
|
var compressed = Compress(body);
|
|
var result = new byte[compressed.Length + checksumSize];
|
|
compressed.CopyTo(result.AsSpan());
|
|
checksum.CopyTo(result.AsSpan(compressed.Length));
|
|
return result;
|
|
}
|
|
|
|
public static byte[] DecompressWithTrailingChecksum(ReadOnlySpan<byte> data, int checksumSize)
|
|
{
|
|
if (data.Length <= checksumSize)
|
|
return data.ToArray();
|
|
|
|
var body = data[..^checksumSize];
|
|
var checksum = data[^checksumSize..];
|
|
|
|
var decompressed = Decompress(body);
|
|
var result = new byte[decompressed.Length + checksumSize];
|
|
decompressed.CopyTo(result.AsSpan());
|
|
checksum.CopyTo(result.AsSpan(decompressed.Length));
|
|
return result;
|
|
}
|
|
}
|
|
```
|
|
|
|
**Step 6: Implement AeadEncryptor**
|
|
|
|
```csharp
|
|
// src/NATS.Server/JetStream/Storage/AeadEncryptor.cs
|
|
using System.Security.Cryptography;
|
|
|
|
namespace NATS.Server.JetStream.Storage;
|
|
|
|
// Go reference: filestore.go lines 800-873 (genEncryptionKey)
|
|
// Matches Go: ChaCha20-Poly1305 (primary), AES-GCM (fallback)
|
|
public enum StoreCipher : byte
|
|
{
|
|
NoCipher = 0,
|
|
ChaCha = 1, // Go: ChaCha StoreCipher = iota
|
|
AesGcm = 2, // Go: AES
|
|
}
|
|
|
|
public static class AeadEncryptor
|
|
{
|
|
private const int NonceSize = 12;
|
|
private const int TagSize = 16;
|
|
|
|
// Wire format: [12:nonce][16:tag][N:ciphertext]
|
|
public static byte[] Encrypt(ReadOnlySpan<byte> plaintext, ReadOnlySpan<byte> key, StoreCipher cipher)
|
|
{
|
|
var nonce = new byte[NonceSize];
|
|
RandomNumberGenerator.Fill(nonce);
|
|
var ciphertext = new byte[plaintext.Length];
|
|
var tag = new byte[TagSize];
|
|
|
|
if (cipher == StoreCipher.ChaCha)
|
|
{
|
|
using var chacha = new ChaCha20Poly1305(key);
|
|
chacha.Encrypt(nonce, plaintext, ciphertext, tag);
|
|
}
|
|
else
|
|
{
|
|
using var aes = new AesGcm(key, TagSize);
|
|
aes.Encrypt(nonce, plaintext, ciphertext, tag);
|
|
}
|
|
|
|
var result = new byte[NonceSize + TagSize + ciphertext.Length];
|
|
nonce.CopyTo(result.AsSpan(0, NonceSize));
|
|
tag.CopyTo(result.AsSpan(NonceSize, TagSize));
|
|
ciphertext.CopyTo(result.AsSpan(NonceSize + TagSize));
|
|
return result;
|
|
}
|
|
|
|
public static byte[] Decrypt(ReadOnlySpan<byte> encrypted, ReadOnlySpan<byte> key, StoreCipher cipher)
|
|
{
|
|
if (encrypted.Length < NonceSize + TagSize)
|
|
throw new CryptographicException("Encrypted data too short");
|
|
|
|
var nonce = encrypted[..NonceSize];
|
|
var tag = encrypted[NonceSize..(NonceSize + TagSize)];
|
|
var ciphertext = encrypted[(NonceSize + TagSize)..];
|
|
var plaintext = new byte[ciphertext.Length];
|
|
|
|
if (cipher == StoreCipher.ChaCha)
|
|
{
|
|
using var chacha = new ChaCha20Poly1305(key);
|
|
chacha.Decrypt(nonce, ciphertext, tag, plaintext);
|
|
}
|
|
else
|
|
{
|
|
using var aes = new AesGcm(key, TagSize);
|
|
aes.Decrypt(nonce, ciphertext, tag, plaintext);
|
|
}
|
|
|
|
return plaintext;
|
|
}
|
|
}
|
|
```
|
|
|
|
**Step 7: Add StoreCipher enum to FileStoreOptions**
|
|
|
|
Modify `src/NATS.Server/JetStream/Storage/FileStoreOptions.cs` to add:
|
|
```csharp
|
|
public StoreCipher Cipher { get; set; } = StoreCipher.NoCipher;
|
|
public StoreCompression Compression { get; set; } = StoreCompression.NoCompression;
|
|
```
|
|
|
|
Add compression enum:
|
|
```csharp
|
|
public enum StoreCompression : byte
|
|
{
|
|
NoCompression = 0,
|
|
S2Compression = 1,
|
|
}
|
|
```
|
|
|
|
**Step 8: Run tests**
|
|
|
|
Run: `dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~S2CodecTests|FullyQualifiedName~AeadEncryptorTests" -v normal`
|
|
Expected: PASS (6 tests)
|
|
|
|
**Step 9: Integrate S2 + AEAD into FileStore**
|
|
|
|
Modify `FileStore.cs` `TransformForPersist` and `RestorePayload` to use the new codecs based on `_options.Cipher` and `_options.Compression`. Keep legacy FSV1 read support.
|
|
|
|
**Step 10: Run full test suite**
|
|
|
|
Run: `dotnet test tests/NATS.Server.Tests -v normal`
|
|
Expected: All existing tests still pass (no regressions)
|
|
|
|
**Step 11: Commit**
|
|
|
|
```bash
|
|
git add Directory.Packages.props src/NATS.Server/NATS.Server.csproj \
|
|
src/NATS.Server/JetStream/Storage/S2Codec.cs \
|
|
src/NATS.Server/JetStream/Storage/AeadEncryptor.cs \
|
|
src/NATS.Server/JetStream/Storage/FileStore.cs \
|
|
src/NATS.Server/JetStream/Storage/FileStoreOptions.cs \
|
|
tests/NATS.Server.Tests/JetStream/Storage/S2CodecTests.cs \
|
|
tests/NATS.Server.Tests/JetStream/Storage/AeadEncryptorTests.cs
|
|
git commit -m "feat: add S2 compression and AEAD encryption for FileStore (Go parity)"
|
|
```
|
|
|
|
---
|
|
|
|
## Phase 2: High Priority Test Ports
|
|
|
|
### Task 5: JetStream Cluster Test Infrastructure
|
|
|
|
**Files:**
|
|
- Create: `tests/NATS.Server.Tests/JetStream/Cluster/JetStreamClusterFixture.cs`
|
|
|
|
**Context:** Go's `jetstream_helpers_test.go` provides `createJetStreamClusterExplicit()`, `cluster` struct with 30+ helper methods, and `supercluster` for multi-cluster scenarios. The .NET equivalent needs to spin up 3+ NATS server instances with JetStream enabled, wait for meta-leader election, and provide synchronization helpers.
|
|
|
|
**Step 1: Write the cluster fixture**
|
|
|
|
```csharp
|
|
// tests/NATS.Server.Tests/JetStream/Cluster/JetStreamClusterFixture.cs
|
|
using NATS.Server.Configuration;
|
|
|
|
namespace NATS.Server.Tests.JetStream.Cluster;
|
|
|
|
// Go reference: jetstream_helpers_test.go — cluster struct + helpers
|
|
internal sealed class JetStreamClusterFixture : IAsyncDisposable
|
|
{
|
|
private readonly List<(NatsServer Server, CancellationTokenSource Cts)> _nodes = [];
|
|
|
|
public IReadOnlyList<NatsServer> Servers => _nodes.Select(n => n.Server).ToList();
|
|
public int NodeCount => _nodes.Count;
|
|
|
|
private JetStreamClusterFixture() { }
|
|
|
|
public static async Task<JetStreamClusterFixture> StartAsync(
|
|
int nodeCount = 3, string clusterName = "test-cluster")
|
|
{
|
|
var fixture = new JetStreamClusterFixture();
|
|
var routePort = GetFreePort();
|
|
|
|
for (var i = 0; i < nodeCount; i++)
|
|
{
|
|
var opts = new NatsOptions
|
|
{
|
|
Host = "127.0.0.1",
|
|
Port = 0,
|
|
ServerName = $"{clusterName}-n{i}",
|
|
Cluster = new ClusterOptions
|
|
{
|
|
Name = clusterName,
|
|
Host = "127.0.0.1",
|
|
Port = i == 0 ? routePort : 0,
|
|
Routes = [$"127.0.0.1:{routePort}"],
|
|
},
|
|
JetStream = new JetStreamOptions
|
|
{
|
|
StoreDir = Path.Combine(Path.GetTempPath(),
|
|
$"nats-js-cluster-{Guid.NewGuid():N}", $"n{i}"),
|
|
},
|
|
};
|
|
|
|
var cts = new CancellationTokenSource();
|
|
var server = new NatsServer(opts, NullLoggerFactory.Instance);
|
|
await server.StartAsync(cts.Token);
|
|
await server.WaitForReadyAsync();
|
|
fixture._nodes.Add((server, cts));
|
|
}
|
|
|
|
await fixture.WaitForClusterFormedAsync();
|
|
return fixture;
|
|
}
|
|
|
|
// Go ref: checkClusterFormed — wait for all nodes to see expected route count
|
|
public async Task WaitForClusterFormedAsync(int timeoutSeconds = 10)
|
|
{
|
|
using var timeout = new CancellationTokenSource(TimeSpan.FromSeconds(timeoutSeconds));
|
|
var expectedRoutes = _nodes.Count - 1;
|
|
|
|
while (!timeout.IsCancellationRequested)
|
|
{
|
|
var allFormed = _nodes.All(n =>
|
|
Interlocked.Read(ref n.Server.Stats.Routes) >= expectedRoutes);
|
|
if (allFormed) return;
|
|
await Task.Delay(50, timeout.Token).ConfigureAwait(ConfigureAwaitOptions.SuppressThrowing);
|
|
}
|
|
|
|
throw new TimeoutException($"Cluster did not form within {timeoutSeconds}s");
|
|
}
|
|
|
|
public async ValueTask DisposeAsync()
|
|
{
|
|
foreach (var (server, cts) in _nodes)
|
|
{
|
|
await cts.CancelAsync();
|
|
server.Dispose();
|
|
cts.Dispose();
|
|
}
|
|
_nodes.Clear();
|
|
}
|
|
|
|
private static int GetFreePort()
|
|
{
|
|
using var socket = new System.Net.Sockets.Socket(
|
|
System.Net.Sockets.AddressFamily.InterNetwork,
|
|
System.Net.Sockets.SocketType.Stream,
|
|
System.Net.Sockets.ProtocolType.Tcp);
|
|
socket.Bind(new System.Net.IPEndPoint(System.Net.IPAddress.Loopback, 0));
|
|
return ((System.Net.IPEndPoint)socket.LocalEndPoint!).Port;
|
|
}
|
|
}
|
|
```
|
|
|
|
**Step 2: Write a smoke test for the fixture**
|
|
|
|
```csharp
|
|
[Fact]
|
|
public async Task Three_node_cluster_forms_and_all_nodes_see_routes()
|
|
{
|
|
await using var cluster = await JetStreamClusterFixture.StartAsync(3);
|
|
cluster.NodeCount.ShouldBe(3);
|
|
foreach (var server in cluster.Servers)
|
|
Interlocked.Read(ref server.Stats.Routes).ShouldBeGreaterThanOrEqualTo(2);
|
|
}
|
|
```
|
|
|
|
**Step 3: Run, verify, commit**
|
|
|
|
Run: `dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamClusterFixture" -v normal`
|
|
|
|
```bash
|
|
git add tests/NATS.Server.Tests/JetStream/Cluster/JetStreamClusterFixture.cs
|
|
git commit -m "feat: add JetStreamClusterFixture for multi-node cluster tests"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 6: JetStream Cluster Tests — Leader Election & Failover (~80 tests)
|
|
|
|
**Files:**
|
|
- Create: `tests/NATS.Server.Tests/JetStream/Cluster/JsClusterLeaderElectionTests.cs`
|
|
- Create: `tests/NATS.Server.Tests/JetStream/Cluster/JsClusterFailoverTests.cs`
|
|
|
|
**Go reference:** `jetstream_cluster_1_test.go` and `jetstream_cluster_2_test.go` — tests for meta-leader election, stream leader selection, leader stepdown, and failover recovery.
|
|
|
|
**Representative test pattern:**
|
|
|
|
```csharp
|
|
[Fact]
|
|
public async Task Meta_leader_elected_in_three_node_cluster()
|
|
{
|
|
await using var cluster = await JetStreamClusterFixture.StartAsync(3);
|
|
// Go ref: TestJetStreamClusterMetaLeaderElection
|
|
// Verify exactly one meta-leader exists
|
|
var leaders = cluster.Servers.Count(s => s.JetStreamIsLeader);
|
|
leaders.ShouldBe(1);
|
|
}
|
|
```
|
|
|
|
Port ~80 tests covering: meta-leader election, stream leader placement, consumer leader selection, leader stepdown API, forced failover, split-brain prevention, term validation, peer removal.
|
|
|
|
**Use parallel subagent**: This task is independent and can run as a sonnet subagent.
|
|
|
|
---
|
|
|
|
### Task 7: JetStream Cluster Tests — Stream Replication (~100 tests)
|
|
|
|
**Files:**
|
|
- Create: `tests/NATS.Server.Tests/JetStream/Cluster/JsClusterStreamReplicationTests.cs`
|
|
- Create: `tests/NATS.Server.Tests/JetStream/Cluster/JsClusterStreamPlacementTests.cs`
|
|
|
|
**Go reference:** `jetstream_cluster_1_test.go` and `jetstream_cluster_3_test.go` — tests for R1/R3 replication, stream placement preferences, peer removal, mirror/source across clusters.
|
|
|
|
Port ~100 tests covering: replicated stream creation, R1 vs R3 behavior, placement tags, peer removal and rebalancing, stream info consistency across nodes, mirror catchup, source aggregation.
|
|
|
|
**Use parallel subagent**: Independent of Task 6.
|
|
|
|
---
|
|
|
|
### Task 8: JetStream Cluster Tests — Consumer Replication (~80 tests)
|
|
|
|
**Files:**
|
|
- Create: `tests/NATS.Server.Tests/JetStream/Cluster/JsClusterConsumerReplicationTests.cs`
|
|
|
|
**Go reference:** `jetstream_cluster_2_test.go` and `jetstream_cluster_4_test.go` — tests for consumer state replication, ack tracking across failover, push consumer reconnection.
|
|
|
|
Port ~80 tests covering: consumer state after leader change, pending ack survival, redelivery after failover, pull consumer batch state, push consumer reconnection, consumer pause/resume.
|
|
|
|
**Use parallel subagent**: Independent of Tasks 6-7.
|
|
|
|
---
|
|
|
|
### Task 9: JetStream Cluster Tests — Meta-cluster Governance (~60 tests)
|
|
|
|
**Files:**
|
|
- Create: `tests/NATS.Server.Tests/JetStream/Cluster/JsClusterMetaGovernanceTests.cs`
|
|
|
|
**Go reference:** `jetstream_cluster_3_test.go` and `jetstream_cluster_4_test.go` — tests for meta-cluster consistency, account limit enforcement in clustered mode, API responses from non-leaders.
|
|
|
|
Port ~60 tests covering: meta-cluster peer count, account limits across cluster, stream create from non-leader, consumer create from non-leader, server removal from meta-group.
|
|
|
|
---
|
|
|
|
### Task 10: JetStream Cluster Tests — Advanced & Long-running (~40 tests)
|
|
|
|
**Files:**
|
|
- Create: `tests/NATS.Server.Tests/JetStream/Cluster/JsClusterAdvancedTests.cs`
|
|
- Create: `tests/NATS.Server.Tests/JetStream/Cluster/JsClusterLongRunningTests.cs`
|
|
|
|
**Go reference:** `jetstream_cluster_4_test.go` and `jetstream_cluster_long_test.go`
|
|
|
|
Port ~40 tests covering: super-cluster scenarios, gateway + JetStream interaction, domain-scoped API, long-running stability (mark as `[Trait("Category", "LongRunning")]`).
|
|
|
|
---
|
|
|
|
### Task 11: JetStream Core Tests (~100 tests)
|
|
|
|
**Files:**
|
|
- Modify: `tests/NATS.Server.Tests/JetStream/` (multiple existing files)
|
|
|
|
**Go reference:** `jetstream_test.go` (312 tests, ~200 already ported)
|
|
|
|
Port remaining ~100 tests covering:
|
|
- Stream lifecycle: max messages, max bytes, max age, discard old/new policy
|
|
- Consumer semantics: ack wait, max deliver, backoff, idle heartbeat
|
|
- Publish preconditions: expected stream, expected seq, expected msg ID, dedup window
|
|
- Account limits: max streams per account, max consumers, max storage bytes
|
|
- API error shapes: exact error codes matching Go's `NewJSXxxError()` responses
|
|
- Direct get: zero-copy message retrieval by sequence and by last-per-subject
|
|
|
|
**Use parallel subagent**: Independent of cluster tests.
|
|
|
|
---
|
|
|
|
### Task 12: FileStore Permutation Tests (~100 tests)
|
|
|
|
**Files:**
|
|
- Create: `tests/NATS.Server.Tests/JetStream/Storage/FileStorePermutationTests.cs`
|
|
- Modify existing: `FileStoreCompressionTests.cs`, `FileStoreEncryptionTests.cs`
|
|
|
|
**Go reference:** `filestore_test.go` (232 tests, ~130 already ported). Go's `testFileStoreAllPermutations()` runs each test across 6 combinations: `{NoCipher, ChaCha, AES} x {NoCompression, S2}`.
|
|
|
|
**Permutation helper:**
|
|
|
|
```csharp
|
|
// Go ref: filestore_test.go lines 55-71 (testFileStoreAllPermutations)
|
|
public static IEnumerable<object[]> AllPermutations()
|
|
{
|
|
foreach (var cipher in new[] { StoreCipher.NoCipher, StoreCipher.ChaCha, StoreCipher.AesGcm })
|
|
foreach (var compression in new[] { StoreCompression.NoCompression, StoreCompression.S2Compression })
|
|
yield return [cipher, compression];
|
|
}
|
|
|
|
[Theory]
|
|
[MemberData(nameof(AllPermutations))]
|
|
public async Task Store_and_load_basic(StoreCipher cipher, StoreCompression compression)
|
|
{
|
|
await using var store = CreateStore($"basic-{cipher}-{compression}", cipher, compression);
|
|
var seq = await store.AppendAsync("test.subject", "hello"u8.ToArray(), CancellationToken.None);
|
|
var msg = await store.LoadAsync(seq, CancellationToken.None);
|
|
msg.ShouldNotBeNull();
|
|
msg!.Payload.ToArray().ShouldBe("hello"u8.ToArray());
|
|
}
|
|
```
|
|
|
|
Port ~100 tests covering: basic CRUD across all 6 permutations, block rotation, crash recovery, corruption detection, large payloads, subject-filtered queries, purge, snapshot/restore.
|
|
|
|
**Use parallel subagent**: Independent of cluster tests.
|
|
|
|
---
|
|
|
|
## Phase 3: Medium/Low Priority Test Ports
|
|
|
|
### Task 13: Stress/NoRace Tests (~50 tests)
|
|
|
|
**Files:**
|
|
- Create: `tests/NATS.Server.Tests/Stress/ConcurrentPubSubStressTests.cs`
|
|
- Create: `tests/NATS.Server.Tests/Stress/SlowConsumerStressTests.cs`
|
|
- Create: `tests/NATS.Server.Tests/Stress/ClusterStressTests.cs`
|
|
|
|
**Go reference:** `norace_1_test.go` (100 tests), `norace_2_test.go` (41 tests)
|
|
|
|
All tests marked with `[Trait("Category", "Stress")]` for optional CI execution.
|
|
|
|
Port ~50 most critical tests covering: concurrent pub/sub with 100+ clients, slow consumer handling under load, route/gateway reconnection under message flood, JetStream publish during cluster failover.
|
|
|
|
---
|
|
|
|
### Task 14: Accounts/Auth Tests (~30 tests)
|
|
|
|
**Files:**
|
|
- Modify: `tests/NATS.Server.Tests/Accounts/` (existing files)
|
|
- Create: `tests/NATS.Server.Tests/Accounts/AuthCalloutTests.cs`
|
|
|
|
**Go reference:** `accounts_test.go` (64 tests), `auth_callout_test.go` (31 tests)
|
|
|
|
Port ~30 remaining tests covering: service import/export cross-account delivery, auth callout timeout/retry, account connection/subscription limits, user revocation.
|
|
|
|
---
|
|
|
|
### Task 15: Message Trace Tests (~20 tests)
|
|
|
|
**Files:**
|
|
- Create: `tests/NATS.Server.Tests/MessageTraceTests.cs`
|
|
|
|
**Go reference:** `msgtrace_test.go` (33 tests)
|
|
|
|
Port ~20 tests covering: trace header propagation, `$SYS.TRACE.>` event publication, trace filtering.
|
|
|
|
---
|
|
|
|
### Task 16: Config/Reload Tests (~20 tests)
|
|
|
|
**Files:**
|
|
- Modify: `tests/NATS.Server.Tests/Configuration/` (existing files)
|
|
|
|
**Go reference:** `opts_test.go` (86 tests), `reload_test.go` (73 tests)
|
|
|
|
Port ~20 remaining tests covering: CLI override precedence, include file resolution, TLS cert reload, account resolver reload.
|
|
|
|
---
|
|
|
|
### Task 17: Events Tests (~15 tests)
|
|
|
|
**Files:**
|
|
- Create: `tests/NATS.Server.Tests/Events/ServerEventTests.cs`
|
|
|
|
**Go reference:** `events_test.go` (51 tests)
|
|
|
|
Port ~15 tests covering: server lifecycle events, account stats, advisory messages.
|
|
|
|
---
|
|
|
|
## Execution Dependencies
|
|
|
|
```
|
|
Task 1 (Wire Format) → Task 2 (NatsRaftTransport)
|
|
Task 3 (JetStream Svc) — independent of Tasks 1-2
|
|
Task 4 (FileStore S2/AEAD) — independent of Tasks 1-3
|
|
|
|
Phase 1 complete → Phase 2 begins
|
|
|
|
Task 5 (Cluster Fixture) → Tasks 6-10 (Cluster Tests, parallel)
|
|
Task 11 (JS Core Tests) — parallel with Tasks 6-10
|
|
Task 12 (FileStore Tests) — parallel with Tasks 6-11
|
|
|
|
Phase 2 complete → Phase 3 begins
|
|
|
|
Tasks 13-17 — all parallel with each other
|
|
```
|
|
|
|
## Verification Checkpoints
|
|
|
|
After each phase:
|
|
|
|
1. `dotnet build` — zero errors, zero warnings
|
|
2. `dotnet test tests/NATS.Server.Tests` — all tests pass
|
|
3. Commit with descriptive message
|
|
4. Verify test count increases as expected
|
|
|
|
**Phase 1 target:** 2,606 + ~20 new impl tests = ~2,626 tests
|
|
**Phase 2 target:** 2,626 + ~460 cluster/JS/FileStore tests = ~3,086 tests
|
|
**Phase 3 target:** 3,086 + ~135 stress/auth/trace/config/events tests = ~3,221 tests
|
|
|
|
**Final success criterion:** 3,100+ tests, 0 failures, all Go subsystems covered.
|