Fix E2E test gaps and add comprehensive E2E + parity test suites
- Fix pull consumer fetch: send original stream subject in HMSG (not inbox) so NATS client distinguishes data messages from control messages - Fix MaxAge expiry: add background timer in StreamManager for periodic pruning - Fix JetStream wire format: Go-compatible anonymous objects with string enums, proper offset-based pagination for stream/consumer list APIs - Add 42 E2E black-box tests (core messaging, auth, TLS, accounts, JetStream) - Add ~1000 parity tests across all subsystems (gaps closure) - Update gap inventory docs to reflect implementation status
This commit is contained in:
Joseph Doherty
166
docs/plans/2026-03-12-e2e-tests-plan.md
Normal file
166
docs/plans/2026-03-12-e2e-tests-plan.md
Normal file
@@ -0,0 +1,166 @@
|
||||
# NATS.E2E.Tests — Implementation Plan
|
||||
|
||||
**Date:** 2026-03-12
|
||||
**Design:** [2026-03-12-e2e-tests-design.md](2026-03-12-e2e-tests-design.md)
|
||||
|
||||
## Steps
|
||||
|
||||
### Step 1: Create the project and add to solution
|
||||
|
||||
**Files:**
|
||||
- `tests/NATS.E2E.Tests/NATS.E2E.Tests.csproj` (new)
|
||||
- `NatsDotNet.slnx` (edit)
|
||||
|
||||
**Details:**
|
||||
|
||||
Create `tests/NATS.E2E.Tests/NATS.E2E.Tests.csproj`:
|
||||
```xml
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
|
||||
<PropertyGroup>
|
||||
<IsPackable>false</IsPackable>
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="coverlet.collector" />
|
||||
<PackageReference Include="Microsoft.NET.Test.Sdk" />
|
||||
<PackageReference Include="NATS.Client.Core" />
|
||||
<PackageReference Include="Shouldly" />
|
||||
<PackageReference Include="xunit" />
|
||||
<PackageReference Include="xunit.runner.visualstudio" />
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<Using Include="Xunit" />
|
||||
<Using Include="Shouldly" />
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
```
|
||||
|
||||
No project references — black-box only. All package versions come from `Directory.Packages.props` (CPM). TFM inherited from `Directory.Build.props` (`net10.0`).
|
||||
|
||||
Add to `NatsDotNet.slnx` under the `/tests/` folder:
|
||||
```xml
|
||||
<Project Path="tests/NATS.E2E.Tests/NATS.E2E.Tests.csproj" />
|
||||
```
|
||||
|
||||
**Verify:** `dotnet build tests/NATS.E2E.Tests` succeeds.
|
||||
|
||||
---
|
||||
|
||||
### Step 2: Implement `NatsServerProcess`
|
||||
|
||||
**Files:**
|
||||
- `tests/NATS.E2E.Tests/Infrastructure/NatsServerProcess.cs` (new)
|
||||
|
||||
**Details:**
|
||||
|
||||
Class `NatsServerProcess : IAsyncDisposable`:
|
||||
|
||||
- **Constructor**: Takes no args. Allocates a free TCP port via ephemeral socket bind.
|
||||
- **`StartAsync()`**:
|
||||
1. Resolves the host DLL path: walk up from `AppContext.BaseDirectory` to find the solution root (contains `NatsDotNet.slnx`), then build path `src/NATS.Server.Host/bin/Debug/net10.0/NATS.Server.Host.dll`. If not found, run `dotnet build src/NATS.Server.Host/NATS.Server.Host.csproj -c Debug` from solution root first.
|
||||
2. Launch `dotnet exec <dll> -p <port>` via `System.Diagnostics.Process`. Redirect stdout/stderr, capture into `StringBuilder` for diagnostics.
|
||||
3. Poll TCP connect to `127.0.0.1:<port>` every 100ms, timeout after 10s. Throw `TimeoutException` with captured output if server doesn't become ready.
|
||||
- **`DisposeAsync()`**:
|
||||
1. If process is running: try `Process.Kill(entireProcessTree: true)` (cross-platform in .NET 10).
|
||||
2. Wait for exit with 5s timeout, then force kill if still alive.
|
||||
3. Dispose the process.
|
||||
- **Properties**: `int Port`, `string Output` (captured stdout+stderr for diagnostics).
|
||||
|
||||
**Verify:** Builds without errors.
|
||||
|
||||
---
|
||||
|
||||
### Step 3: Implement `NatsServerFixture`
|
||||
|
||||
**Files:**
|
||||
- `tests/NATS.E2E.Tests/Infrastructure/NatsServerFixture.cs` (new)
|
||||
|
||||
**Details:**
|
||||
|
||||
Class `NatsServerFixture : IAsyncLifetime`:
|
||||
|
||||
- **Field**: `NatsServerProcess _server`
|
||||
- **`InitializeAsync()`**: Create `NatsServerProcess`, call `StartAsync()`.
|
||||
- **`DisposeAsync()`**: Dispose the server process.
|
||||
- **`int Port`**: Delegates to `_server.Port`.
|
||||
- **`NatsConnection CreateClient()`**: Returns `new NatsConnection(new NatsOpts { Url = $"nats://127.0.0.1:{Port}" })`.
|
||||
|
||||
Define a collection attribute:
|
||||
```csharp
|
||||
[CollectionDefinition("E2E")]
|
||||
public class E2ECollection : ICollectionFixture<NatsServerFixture>;
|
||||
```
|
||||
|
||||
This lets multiple test classes share one server process via `[Collection("E2E")]`.
|
||||
|
||||
**Verify:** Builds without errors.
|
||||
|
||||
---
|
||||
|
||||
### Step 4: Implement `BasicTests`
|
||||
|
||||
**Files:**
|
||||
- `tests/NATS.E2E.Tests/BasicTests.cs` (new)
|
||||
|
||||
**Details:**
|
||||
|
||||
```csharp
|
||||
[Collection("E2E")]
|
||||
public class BasicTests(NatsServerFixture fixture)
|
||||
{
|
||||
private static CancellationToken Timeout => new CancellationTokenSource(TimeSpan.FromSeconds(10)).Token;
|
||||
```
|
||||
|
||||
**Test 1 — `ConnectAndPing`:**
|
||||
- Create client via `fixture.CreateClient()`
|
||||
- `await client.ConnectAsync()`
|
||||
- `await client.PingAsync()` — if no exception, PING/PONG succeeded
|
||||
- Assert `client.ConnectionState` is `Open` (via Shouldly)
|
||||
|
||||
**Test 2 — `PubSub`:**
|
||||
- Create two clients (pub, sub)
|
||||
- Connect both
|
||||
- Subscribe sub to `"e2e.test.pubsub"`
|
||||
- Flush sub via `PingAsync()`
|
||||
- Publish `"hello e2e"` on the subject
|
||||
- Read one message from subscription with 10s timeout
|
||||
- Assert `msg.Data.ShouldBe("hello e2e")`
|
||||
|
||||
**Test 3 — `RequestReply`:**
|
||||
- Create two clients (requester, responder)
|
||||
- Connect both
|
||||
- Subscribe responder to `"e2e.test.rpc"`, in a background task read messages and reply with `"reply: " + msg.Data`
|
||||
- Flush responder via `PingAsync()`
|
||||
- Send request from requester: `await requester.RequestAsync<string, string>("e2e.test.rpc", "ping")`
|
||||
- Assert reply data is `"reply: ping"`
|
||||
|
||||
All tests use `await using` for client cleanup.
|
||||
|
||||
**Verify:** `dotnet test tests/NATS.E2E.Tests` — all 3 tests pass.
|
||||
|
||||
---
|
||||
|
||||
### Step 5: Verify full solution builds and tests pass
|
||||
|
||||
**Commands:**
|
||||
```bash
|
||||
dotnet build
|
||||
dotnet test tests/NATS.E2E.Tests -v normal
|
||||
```
|
||||
|
||||
**Success criteria:** Solution builds clean, all 3 E2E tests pass.
|
||||
|
||||
## Batch Structure
|
||||
|
||||
All 5 steps are in a single batch — the project is small and sequential (each step builds on the prior). No parallelization needed.
|
||||
|
||||
| Step | Files | Depends On |
|
||||
|------|-------|------------|
|
||||
| 1 | csproj + slnx | — |
|
||||
| 2 | NatsServerProcess.cs | Step 1 |
|
||||
| 3 | NatsServerFixture.cs | Step 2 |
|
||||
| 4 | BasicTests.cs | Step 3 |
|
||||
| 5 | (verify only) | Step 4 |
|
||||
Reference in New Issue
Block a user