feat: add benchmark test project for Go vs .NET server comparison
Side-by-side performance benchmarks using NATS.Client.Core against both servers on ephemeral ports. Includes core pub/sub, request/reply latency, and JetStream throughput tests with comparison output and benchmarks_comparison.md results. Also fixes timestamp flakiness in StoreInterfaceTests by using explicit timestamps.
This commit is contained in:
Joseph Doherty
212
AGENTS.md
Normal file
212
AGENTS.md
Normal file
@@ -0,0 +1,212 @@
|
||||
# AGENTS.md
|
||||
|
||||
This file provides guidance to Codex (Codex.ai/code) when working with code in this repository.
|
||||
|
||||
## Project Overview
|
||||
|
||||
This project ports the [NATS server](https://github.com/nats-io/nats-server) from Go to .NET 10 / C#. The Go reference implementation lives in `golang/nats-server/`. The .NET port lives at the repository root.
|
||||
|
||||
NATS is a high-performance publish-subscribe messaging system. It supports wildcards (`*` single token, `>` multi-token), queue groups for load balancing, request-reply, clustering (full-mesh routes, gateways, leaf nodes), and persistent streaming via JetStream.
|
||||
|
||||
## Build & Test Commands
|
||||
|
||||
The solution file is `NatsDotNet.slnx`.
|
||||
|
||||
```bash
|
||||
# Build the solution
|
||||
dotnet build
|
||||
|
||||
# Run all tests
|
||||
dotnet test
|
||||
|
||||
# Run tests with verbose output
|
||||
dotnet test -v normal
|
||||
|
||||
# Run a single test project
|
||||
dotnet test tests/NATS.Server.Tests
|
||||
|
||||
# Run a specific test project
|
||||
dotnet test tests/NATS.Server.Core.Tests
|
||||
dotnet test tests/NATS.Server.JetStream.Tests
|
||||
|
||||
# Run a specific test by name
|
||||
dotnet test tests/NATS.Server.Core.Tests --filter "FullyQualifiedName~TestName"
|
||||
|
||||
# Run the NATS server (default port 4222)
|
||||
dotnet run --project src/NATS.Server.Host
|
||||
|
||||
# Run the NATS server on a custom port
|
||||
dotnet run --project src/NATS.Server.Host -- -p 14222
|
||||
|
||||
# Clean and rebuild
|
||||
dotnet clean && dotnet build
|
||||
```
|
||||
|
||||
## .NET Project Structure
|
||||
|
||||
```
|
||||
NatsDotNet.slnx # Solution file
|
||||
src/
|
||||
NATS.Server/ # Core server library
|
||||
NatsServer.cs # Server: listener, accept loop, shutdown
|
||||
NatsClient.cs # Per-connection client: read/write loops, sub tracking
|
||||
NatsOptions.cs # Server configuration (port, host, etc.)
|
||||
Protocol/
|
||||
NatsParser.cs # Protocol state machine (PUB, SUB, UNSUB, etc.)
|
||||
NatsProtocol.cs # Wire-level protocol writing (INFO, MSG, PING/PONG)
|
||||
Subscriptions/
|
||||
SubjectMatch.cs # Subject validation and wildcard matching
|
||||
SubList.cs # Trie-based subscription list with caching
|
||||
SubListResult.cs # Match result container (plain subs + queue groups)
|
||||
Subscription.cs # Subscription model (subject, sid, queue, client)
|
||||
NATS.Server.Host/ # Executable host app
|
||||
Program.cs # Entry point, CLI arg parsing (-p port)
|
||||
tests/
|
||||
NATS.Server.TestUtilities/ # Shared helpers, fixtures, parity tools (class library)
|
||||
NATS.Server.Core.Tests/ # Client, server, parser, config, subscriptions, protocol
|
||||
NATS.Server.Auth.Tests/ # Auth, accounts, permissions, JWT, NKeys
|
||||
NATS.Server.JetStream.Tests/ # JetStream API, streams, consumers, storage, cluster
|
||||
NATS.Server.Raft.Tests/ # RAFT consensus
|
||||
NATS.Server.Clustering.Tests/ # Routes, cluster topology, inter-server protocol
|
||||
NATS.Server.Gateways.Tests/ # Gateway connections, interest modes
|
||||
NATS.Server.LeafNodes.Tests/ # Leaf node connections, hub-spoke
|
||||
NATS.Server.Mqtt.Tests/ # MQTT protocol bridge
|
||||
NATS.Server.Monitoring.Tests/ # Monitor endpoints, events, system events
|
||||
NATS.Server.Transport.Tests/ # WebSocket, TLS, OCSP, IO
|
||||
NATS.E2E.Tests/ # End-to-end tests using NATS.Client.Core NuGet
|
||||
```
|
||||
|
||||
## Go Reference Commands
|
||||
|
||||
```bash
|
||||
# Build the Go reference server
|
||||
cd golang/nats-server && go build
|
||||
|
||||
# Run Go tests for a specific area
|
||||
cd golang/nats-server && go test -v -run TestName ./server/ -count=1 -timeout=30m
|
||||
|
||||
# Run all Go server tests (slow, ~30min)
|
||||
cd golang/nats-server && go test -v ./server/ -count=1 -timeout=30m
|
||||
```
|
||||
|
||||
## Architecture: NATS Server (Reference)
|
||||
|
||||
The Go source in `golang/nats-server/server/` is the authoritative reference. Key files by subsystem:
|
||||
|
||||
### Core Message Path
|
||||
- **`server.go`** — Server struct, startup lifecycle (`NewServer` → `Run` → `WaitForShutdown`), listener management
|
||||
- **`client.go`** (6700 lines) — Connection handling, `readLoop`/`writeLoop` goroutines, per-client subscription tracking, dynamic buffer sizing (512→65536 bytes), client types: `CLIENT`, `ROUTER`, `GATEWAY`, `LEAF`, `SYSTEM`
|
||||
- **`parser.go`** — Protocol state machine. Text protocol: `PUB`, `SUB`, `UNSUB`, `CONNECT`, `INFO`, `PING/PONG`, `MSG`. Extended: `HPUB/HMSG` (headers), `RPUB/RMSG` (routes). Control line limit: 4096 bytes. Default max payload: 1MB.
|
||||
- **`sublist.go`** — Trie-based subject matcher with wildcard support. Nodes have `psubs` (plain), `qsubs` (queue groups), special pointers for `*` and `>` wildcards. Results are cached with atomic generation IDs for invalidation.
|
||||
|
||||
### Authentication & Accounts
|
||||
- **`auth.go`** — Auth mechanisms: username/password, token, NKeys (Ed25519), JWT, external auth callout, LDAP
|
||||
- **`accounts.go`** (137KB) — Multi-tenant account isolation. Each account has its own `Sublist`, client set, and subject namespace. Supports exports/imports between accounts, service latency tracking.
|
||||
- **`jwt.go`**, **`nkey.go`** — JWT claims parsing and NKey validation
|
||||
|
||||
### Clustering
|
||||
- **`route.go`** — Full-mesh cluster routes. Route pooling (default 3 connections per peer). Account-specific dedicated routes. Protocol: `RS+`/`RS-` for subscribe propagation, `RMSG` for routed messages.
|
||||
- **`gateway.go`** (103KB) — Inter-cluster bridges. Interest-only mode optimizes traffic. Reply subject mapping (`_GR_.` prefix) avoids cross-cluster conflicts.
|
||||
- **`leafnode.go`** — Hub-and-spoke topology for edge deployments. Only subscribed subjects shared with hub. Loop detection via `$LDS.` prefix.
|
||||
|
||||
### JetStream (Persistence)
|
||||
- **`jetstream.go`** — Orchestration, API subject handlers (`$JS.API.*`)
|
||||
- **`stream.go`** (8000 lines) — Stream lifecycle, retention policies (Limits, Interest, WorkQueue), subject transforms, mirroring/sourcing
|
||||
- **`consumer.go`** — Stateful readers. Push vs pull delivery. Ack policies: None, All, Explicit. Redelivery tracking, priority groups.
|
||||
- **`filestore.go`** (337KB) — Block-based persistent storage with S2 compression, encryption (ChaCha20/AES-GCM), indexing
|
||||
- **`memstore.go`** — In-memory storage with hash-wheel TTL expiration
|
||||
- **`raft.go`** — RAFT consensus for clustered JetStream. Meta-cluster for metadata, per-stream/consumer RAFT groups.
|
||||
|
||||
### Configuration & Monitoring
|
||||
- **`opts.go`** — CLI flags + config file loading. CLI overrides config. Supports hot reload on signal.
|
||||
- **`monitor.go`** — HTTP endpoints: `/varz`, `/connz`, `/routez`, `/gatewayz`, `/jsz`, `/healthz`
|
||||
- **`conf/`** — Config file parser (custom format with includes)
|
||||
|
||||
### Internal Data Structures
|
||||
- **`server/avl/`** — AVL tree for sparse sequence sets (ack tracking)
|
||||
- **`server/stree/`** — Subject tree for per-subject state in streams
|
||||
- **`server/gsl/`** — Generic subject list, optimized trie
|
||||
- **`server/thw/`** — Time hash wheel for efficient TTL expiration
|
||||
|
||||
## Key Porting Considerations
|
||||
|
||||
**Concurrency model:** Go uses goroutines (one per connection readLoop + writeLoop). Map to async/await with `Task`-based I/O. Use `Channel<T>` or `Pipe` for producer-consumer patterns where Go uses channels.
|
||||
|
||||
**Locking:** Go `sync.RWMutex` maps to `ReaderWriterLockSlim`. Go `sync.Map` maps to `ConcurrentDictionary`. Go `atomic` operations map to `Interlocked` or `volatile`.
|
||||
|
||||
**Subject matching:** The `Sublist` trie is performance-critical. Every published message triggers a `Match()` call. Cache invalidation uses atomic generation counters.
|
||||
|
||||
**Protocol parsing:** The parser is a byte-by-byte state machine. In .NET, use `System.IO.Pipelines` for zero-copy parsing with `ReadOnlySequence<byte>`.
|
||||
|
||||
**Buffer management:** Go uses `[]byte` slices with pooling. Map to `ArrayPool<byte>` and `Memory<T>`/`Span<T>`.
|
||||
|
||||
**Compression:** NATS uses S2 (Snappy variant) for route/gateway compression. Use an equivalent .NET S2 library or IronSnappy.
|
||||
|
||||
**Ports:** Client=4222, Cluster=6222, Monitoring=8222, Leaf=5222, Gateway=7222.
|
||||
|
||||
## Message Flow Summary
|
||||
|
||||
```
|
||||
Client PUB → parser → permission check → Sublist.Match() →
|
||||
├─ Local subscribers: MSG to each (queue subs: pick one per group)
|
||||
├─ Cluster routes: RMSG to peers (who deliver to their locals)
|
||||
├─ Gateways: forward to interested remote clusters
|
||||
└─ JetStream: if subject matches a stream, store + deliver to consumers
|
||||
```
|
||||
|
||||
## NuGet Package Management
|
||||
|
||||
This solution uses **Central Package Management (CPM)** via `Directory.Packages.props` at the repo root. All package versions are defined centrally there.
|
||||
|
||||
- In `.csproj` files, use `<PackageReference Include="Foo" />` **without** a `Version` attribute
|
||||
- To add a new package: add a `<PackageVersion>` entry in `Directory.Packages.props`, then reference it without version in the project's csproj
|
||||
- To update a version: change it only in `Directory.Packages.props` — all projects pick it up automatically
|
||||
- Never specify `Version` on `<PackageReference>` in individual csproj files
|
||||
|
||||
## Logging
|
||||
|
||||
Use **Microsoft.Extensions.Logging** (`ILogger<T>`) for all logging throughout the server. Wire up **Serilog** as the logging provider in the host application.
|
||||
|
||||
- Inject `ILogger<T>` via constructor in all components (NatsServer, NatsClient, etc.)
|
||||
- Use **Serilog.Context.LogContext** to push contextual properties (client ID, remote endpoint, subscription subject) so they appear on all log entries within that scope
|
||||
- Use structured logging with message templates: `logger.LogInformation("Client {ClientId} subscribed to {Subject}", id, subject)` — never string interpolation
|
||||
- Log levels: `Trace` for protocol bytes, `Debug` for per-message flow, `Information` for lifecycle events (connect/disconnect), `Warning` for protocol violations, `Error` for unexpected failures
|
||||
|
||||
## Testing
|
||||
|
||||
- **xUnit 3** for test framework
|
||||
- **Shouldly** for assertions — use `value.ShouldBe(expected)`, `action.ShouldThrow<T>()`, etc. Do NOT use `Assert.*` from xUnit
|
||||
- **NSubstitute** for mocking/substitution when needed
|
||||
- Do **NOT** use FluentAssertions or Moq — these are explicitly excluded
|
||||
- Test project uses global `using Shouldly;`
|
||||
|
||||
## Porting Guidelines
|
||||
|
||||
- Use modern .NET 10 / C# 14 best practices (primary constructors, collection expressions, `field` keyword where stable, file-scoped namespaces, raw string literals, etc.)
|
||||
- Prefer `readonly record struct` for small value types over mutable structs
|
||||
- Use `required` properties and `init` setters for initialization-only state
|
||||
- Use pattern matching and switch expressions where they improve clarity
|
||||
- Prefer `System.Text.Json` source generators for JSON serialization
|
||||
- Use `ValueTask` where appropriate for hot-path async methods
|
||||
|
||||
## Agent Model Guidance
|
||||
|
||||
- **Sonnet** (`model: "sonnet"`) — use for simpler implementation tasks: straightforward file modifications, adding packages, converting assertions, boilerplate code
|
||||
- **Opus** (default) — use for complex tasks, architectural decisions, design work, tricky protocol logic, and code review
|
||||
- **Parallel subagents** — use where tasks are independent and don't touch the same files (e.g., converting test files in parallel, adding packages while updating docs)
|
||||
|
||||
## Documentation
|
||||
|
||||
Follow the documentation rules in [`documentation_rules.md`](documentation_rules.md) for all project documentation. Key points:
|
||||
|
||||
- Documentation lives in `Documentation/` with component subfolders (Protocol, Subscriptions, Server, Configuration, Operations)
|
||||
- Use `PascalCase.md` file names, always specify language on code blocks, use real code snippets (not invented examples)
|
||||
- Update documentation when code changes — see the trigger rules and component map in the rules file
|
||||
- Technical and direct tone, explain "why" not just "what", present tense
|
||||
|
||||
## Conventions
|
||||
|
||||
- Reference the Go implementation file and line when porting a subsystem
|
||||
- Maintain protocol compatibility — the .NET server must interoperate with existing NATS clients and Go servers in a cluster
|
||||
- Use the same configuration file format as the Go server (parsed by `conf/` package)
|
||||
- Match the Go server's monitoring JSON response shapes for tooling compatibility
|
||||
Reference in New Issue
Block a user