Add CLAUDE.md and base server design document
Design covers the minimal NATS server port: pub/sub with wildcards and queue groups over System.IO.Pipelines, targeting .NET 10.
This commit is contained in:
Joseph Doherty
116
CLAUDE.md
Normal file
116
CLAUDE.md
Normal file
@@ -0,0 +1,116 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## Project Overview
|
||||
|
||||
This project ports the [NATS server](https://github.com/nats-io/nats-server) from Go to .NET 10 / C#. The Go reference implementation lives in `golang/nats-server/`. The .NET port lives at the repository root.
|
||||
|
||||
NATS is a high-performance publish-subscribe messaging system. It supports wildcards (`*` single token, `>` multi-token), queue groups for load balancing, request-reply, clustering (full-mesh routes, gateways, leaf nodes), and persistent streaming via JetStream.
|
||||
|
||||
## Build & Test Commands
|
||||
|
||||
```bash
|
||||
# Build the solution
|
||||
dotnet build
|
||||
|
||||
# Run all tests
|
||||
dotnet test
|
||||
|
||||
# Run a single test project
|
||||
dotnet test <path/to/TestProject.csproj>
|
||||
|
||||
# Run a specific test by name
|
||||
dotnet test --filter "FullyQualifiedName~TestClassName.TestMethodName"
|
||||
|
||||
# Run tests with verbose output
|
||||
dotnet test -v normal
|
||||
|
||||
# Clean and rebuild
|
||||
dotnet clean && dotnet build
|
||||
```
|
||||
|
||||
## Go Reference Commands
|
||||
|
||||
```bash
|
||||
# Build the Go reference server
|
||||
cd golang/nats-server && go build
|
||||
|
||||
# Run Go tests for a specific area
|
||||
cd golang/nats-server && go test -v -run TestName ./server/ -count=1 -timeout=30m
|
||||
|
||||
# Run all Go server tests (slow, ~30min)
|
||||
cd golang/nats-server && go test -v ./server/ -count=1 -timeout=30m
|
||||
```
|
||||
|
||||
## Architecture: NATS Server (Reference)
|
||||
|
||||
The Go source in `golang/nats-server/server/` is the authoritative reference. Key files by subsystem:
|
||||
|
||||
### Core Message Path
|
||||
- **`server.go`** — Server struct, startup lifecycle (`NewServer` → `Run` → `WaitForShutdown`), listener management
|
||||
- **`client.go`** (6700 lines) — Connection handling, `readLoop`/`writeLoop` goroutines, per-client subscription tracking, dynamic buffer sizing (512→65536 bytes), client types: `CLIENT`, `ROUTER`, `GATEWAY`, `LEAF`, `SYSTEM`
|
||||
- **`parser.go`** — Protocol state machine. Text protocol: `PUB`, `SUB`, `UNSUB`, `CONNECT`, `INFO`, `PING/PONG`, `MSG`. Extended: `HPUB/HMSG` (headers), `RPUB/RMSG` (routes). Control line limit: 4096 bytes. Default max payload: 1MB.
|
||||
- **`sublist.go`** — Trie-based subject matcher with wildcard support. Nodes have `psubs` (plain), `qsubs` (queue groups), special pointers for `*` and `>` wildcards. Results are cached with atomic generation IDs for invalidation.
|
||||
|
||||
### Authentication & Accounts
|
||||
- **`auth.go`** — Auth mechanisms: username/password, token, NKeys (Ed25519), JWT, external auth callout, LDAP
|
||||
- **`accounts.go`** (137KB) — Multi-tenant account isolation. Each account has its own `Sublist`, client set, and subject namespace. Supports exports/imports between accounts, service latency tracking.
|
||||
- **`jwt.go`**, **`nkey.go`** — JWT claims parsing and NKey validation
|
||||
|
||||
### Clustering
|
||||
- **`route.go`** — Full-mesh cluster routes. Route pooling (default 3 connections per peer). Account-specific dedicated routes. Protocol: `RS+`/`RS-` for subscribe propagation, `RMSG` for routed messages.
|
||||
- **`gateway.go`** (103KB) — Inter-cluster bridges. Interest-only mode optimizes traffic. Reply subject mapping (`_GR_.` prefix) avoids cross-cluster conflicts.
|
||||
- **`leafnode.go`** — Hub-and-spoke topology for edge deployments. Only subscribed subjects shared with hub. Loop detection via `$LDS.` prefix.
|
||||
|
||||
### JetStream (Persistence)
|
||||
- **`jetstream.go`** — Orchestration, API subject handlers (`$JS.API.*`)
|
||||
- **`stream.go`** (8000 lines) — Stream lifecycle, retention policies (Limits, Interest, WorkQueue), subject transforms, mirroring/sourcing
|
||||
- **`consumer.go`** — Stateful readers. Push vs pull delivery. Ack policies: None, All, Explicit. Redelivery tracking, priority groups.
|
||||
- **`filestore.go`** (337KB) — Block-based persistent storage with S2 compression, encryption (ChaCha20/AES-GCM), indexing
|
||||
- **`memstore.go`** — In-memory storage with hash-wheel TTL expiration
|
||||
- **`raft.go`** — RAFT consensus for clustered JetStream. Meta-cluster for metadata, per-stream/consumer RAFT groups.
|
||||
|
||||
### Configuration & Monitoring
|
||||
- **`opts.go`** — CLI flags + config file loading. CLI overrides config. Supports hot reload on signal.
|
||||
- **`monitor.go`** — HTTP endpoints: `/varz`, `/connz`, `/routez`, `/gatewayz`, `/jsz`, `/healthz`
|
||||
- **`conf/`** — Config file parser (custom format with includes)
|
||||
|
||||
### Internal Data Structures
|
||||
- **`server/avl/`** — AVL tree for sparse sequence sets (ack tracking)
|
||||
- **`server/stree/`** — Subject tree for per-subject state in streams
|
||||
- **`server/gsl/`** — Generic subject list, optimized trie
|
||||
- **`server/thw/`** — Time hash wheel for efficient TTL expiration
|
||||
|
||||
## Key Porting Considerations
|
||||
|
||||
**Concurrency model:** Go uses goroutines (one per connection readLoop + writeLoop). Map to async/await with `Task`-based I/O. Use `Channel<T>` or `Pipe` for producer-consumer patterns where Go uses channels.
|
||||
|
||||
**Locking:** Go `sync.RWMutex` maps to `ReaderWriterLockSlim`. Go `sync.Map` maps to `ConcurrentDictionary`. Go `atomic` operations map to `Interlocked` or `volatile`.
|
||||
|
||||
**Subject matching:** The `Sublist` trie is performance-critical. Every published message triggers a `Match()` call. Cache invalidation uses atomic generation counters.
|
||||
|
||||
**Protocol parsing:** The parser is a byte-by-byte state machine. In .NET, use `System.IO.Pipelines` for zero-copy parsing with `ReadOnlySequence<byte>`.
|
||||
|
||||
**Buffer management:** Go uses `[]byte` slices with pooling. Map to `ArrayPool<byte>` and `Memory<T>`/`Span<T>`.
|
||||
|
||||
**Compression:** NATS uses S2 (Snappy variant) for route/gateway compression. Use an equivalent .NET S2 library or IronSnappy.
|
||||
|
||||
**Ports:** Client=4222, Cluster=6222, Monitoring=8222, Leaf=5222, Gateway=7222.
|
||||
|
||||
## Message Flow Summary
|
||||
|
||||
```
|
||||
Client PUB → parser → permission check → Sublist.Match() →
|
||||
├─ Local subscribers: MSG to each (queue subs: pick one per group)
|
||||
├─ Cluster routes: RMSG to peers (who deliver to their locals)
|
||||
├─ Gateways: forward to interested remote clusters
|
||||
└─ JetStream: if subject matches a stream, store + deliver to consumers
|
||||
```
|
||||
|
||||
## Conventions
|
||||
|
||||
- Reference the Go implementation file and line when porting a subsystem
|
||||
- Maintain protocol compatibility — the .NET server must interoperate with existing NATS clients and Go servers in a cluster
|
||||
- Use the same configuration file format as the Go server (parsed by `conf/` package)
|
||||
- Match the Go server's monitoring JSON response shapes for tooling compatibility
|
||||
140
docs/plans/2026-02-22-base-server-design.md
Normal file
140
docs/plans/2026-02-22-base-server-design.md
Normal file
@@ -0,0 +1,140 @@
|
||||
# NATS .NET Base Server Design
|
||||
|
||||
**Date:** 2026-02-22
|
||||
**Scope:** Minimal single-node NATS server — pub/sub, queue groups, wildcards. No auth, clustering, JetStream, or monitoring.
|
||||
**Target:** .NET 10, C#
|
||||
**Approach:** Bottom-up (Sublist → Parser → Client → Server)
|
||||
|
||||
## Decisions
|
||||
|
||||
- **Framework:** .NET 10
|
||||
- **I/O:** System.IO.Pipelines for zero-copy protocol parsing
|
||||
- **Structure:** Library (NATS.Server) + Host (NATS.Server.Host) + Tests (NATS.Server.Tests)
|
||||
- **Test framework:** xUnit
|
||||
- **Queue group selection:** Round-robin with atomic counter
|
||||
|
||||
## Solution Layout
|
||||
|
||||
```
|
||||
natsdotnet/
|
||||
├── NatsDotNet.sln
|
||||
├── src/
|
||||
│ ├── NATS.Server/ # Core library
|
||||
│ │ ├── NatsServer.cs # Server orchestrator
|
||||
│ │ ├── NatsClient.cs # Client connection
|
||||
│ │ ├── NatsOptions.cs # Configuration
|
||||
│ │ ├── Protocol/
|
||||
│ │ │ ├── NatsProtocol.cs # Constants, op codes
|
||||
│ │ │ └── NatsParser.cs # Pipeline-based parser
|
||||
│ │ └── Subscriptions/
|
||||
│ │ ├── Subscription.cs
|
||||
│ │ ├── SubjectMatch.cs
|
||||
│ │ └── SubList.cs # Trie + cache
|
||||
│ └── NATS.Server.Host/ # Console app
|
||||
│ └── Program.cs
|
||||
└── tests/
|
||||
└── NATS.Server.Tests/
|
||||
```
|
||||
|
||||
## Component Designs
|
||||
|
||||
### 1. SubList (subject matching trie)
|
||||
|
||||
Reference: `golang/nats-server/server/sublist.go`
|
||||
|
||||
**Structures:**
|
||||
- `SubList` — root trie + cache (`Dictionary<string, SubListResult>`) + `ReaderWriterLockSlim` + atomic `genId`
|
||||
- `TrieLevel` — `Dictionary<string, TrieNode>` for literal tokens, nullable `pwc` (`*`) and `fwc` (`>`) pointers
|
||||
- `TrieNode` — optional `next: TrieLevel`, `HashSet<Subscription>` for plain subs, `Dictionary<string, HashSet<Subscription>>` for queue subs
|
||||
- `Subscription` — client ref, subject, queue, sid, atomic message count, max messages
|
||||
- `SubListResult` — `Subscription[]` plain subs, `Subscription[][]` queue subs (grouped)
|
||||
|
||||
**Operations:**
|
||||
- `Insert(Subscription)` — walk trie by `.`-split tokens, create nodes, add sub, increment genId (clears cache)
|
||||
- `Remove(Subscription)` — reverse, prune empty nodes, increment genId
|
||||
- `Match(ReadOnlySpan<byte>)` — cache check (convert to string key on miss), walk trie collecting matches from literal + `*` + `>` nodes
|
||||
- Thread safety: read lock for Match, write lock for Insert/Remove
|
||||
|
||||
### 2. Protocol Parser
|
||||
|
||||
Reference: `golang/nats-server/server/parser.go`
|
||||
|
||||
Operates on `PipeReader`. Scans for `\r\n` to delimit control lines. Identifies command by first bytes. For `PUB`/`HPUB`, reads control line then exactly `size` bytes of payload + `\r\n`.
|
||||
|
||||
**Commands (base server):**
|
||||
|
||||
| Command | Direction | Control line format |
|
||||
|---------|-----------|-------------------|
|
||||
| INFO | S→C | `INFO {json}\r\n` |
|
||||
| CONNECT | C→S | `CONNECT {json}\r\n` |
|
||||
| PUB | C→S | `PUB subject [reply] size\r\n[payload]\r\n` |
|
||||
| HPUB | C→S | `HPUB subject [reply] hdr_size total_size\r\n[hdrs+payload]\r\n` |
|
||||
| SUB | C→S | `SUB subject [queue] sid\r\n` |
|
||||
| UNSUB | C→S | `UNSUB sid [max_msgs]\r\n` |
|
||||
| MSG | S→C | `MSG subject sid [reply] size\r\n[payload]\r\n` |
|
||||
| HMSG | S→C | `HMSG subject sid [reply] hdr_size total_size\r\n[hdrs+payload]\r\n` |
|
||||
| PING/PONG | Both | `PING\r\n` / `PONG\r\n` |
|
||||
| +OK / -ERR | S→C | `+OK\r\n` / `-ERR 'msg'\r\n` |
|
||||
|
||||
**Design choices:**
|
||||
- Parser returns parsed command structs, does not dispatch
|
||||
- No string allocation on hot path — subject matching uses `Span<byte>`
|
||||
- Max control line: 4096 bytes, max payload: 1MB default (configurable)
|
||||
|
||||
### 3. NatsClient (connection handler)
|
||||
|
||||
Reference: `golang/nats-server/server/client.go`
|
||||
|
||||
**Fields:** unique id, Socket, IDuplexPipe, back-ref to server, subs dict (sid→Subscription), ClientOptions (from CONNECT json), stats (Interlocked counters), SemaphoreSlim for write serialization.
|
||||
|
||||
**Lifecycle:**
|
||||
1. Server accepts socket, creates NatsClient
|
||||
2. `RunAsync(CancellationToken)`: send INFO → read loop (parse + dispatch commands)
|
||||
3. Dispatch: CONNECT→validate, SUB→insert into SubList, UNSUB→remove, PUB→`server.ProcessMessage()`, PING→PONG
|
||||
4. On disconnect: remove all subs, remove from server clients dict
|
||||
|
||||
**Writing:** `SendMessageAsync()` formats MSG/HMSG, acquires write lock, writes to PipeWriter, flushes.
|
||||
|
||||
### 4. NatsServer (orchestrator)
|
||||
|
||||
Reference: `golang/nats-server/server/server.go`
|
||||
|
||||
**Fields:** NatsOptions, TCP listener Socket, `ConcurrentDictionary<ulong, NatsClient>`, SubList, atomic next client ID, ServerInfo struct, CancellationTokenSource.
|
||||
|
||||
**Core methods:**
|
||||
- `StartAsync()` — bind, accept loop
|
||||
- `ProcessMessage(subject, reply, headers, payload, sender)` — SubList.Match → deliver to plain subs + pick one per queue group (round-robin)
|
||||
- `RemoveClient(client)` — cleanup subs and client dict
|
||||
- `Shutdown()` — cancel, close listener, drain
|
||||
|
||||
### 5. NatsOptions
|
||||
|
||||
Reference: `golang/nats-server/server/opts.go`
|
||||
|
||||
Minimal for base: Host, Port (default 4222), ServerName, MaxPayload (1MB), MaxControlLine (4096), MaxConnections, PingInterval, MaxPingsOut.
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
**Unit tests:**
|
||||
- SubList: insert/remove/match with literals, `*`, `>`, queue groups, cache invalidation. Port Go test cases.
|
||||
- Parser: each command type, malformed input, partial reads, boundary sizes.
|
||||
- Client protocol: CONNECT handshake, SUB/UNSUB lifecycle, PUB delivery, PING/PONG via loopback sockets.
|
||||
|
||||
**Integration tests (using NATS.Client.Core NuGet):**
|
||||
- Basic pub/sub, wildcard matching, queue group distribution, fan-out, UNSUB + auto-unsub, PING/PONG.
|
||||
|
||||
## Module Boundaries (future)
|
||||
|
||||
These are explicitly excluded from this design and will be added as separate modules later:
|
||||
|
||||
| Module | Key files in Go reference |
|
||||
|--------|--------------------------|
|
||||
| Authentication | auth.go, auth_callout.go, nkey.go, jwt.go |
|
||||
| Monitoring HTTP | monitor.go |
|
||||
| Clustering/Routes | route.go |
|
||||
| Gateways | gateway.go |
|
||||
| Leaf Nodes | leafnode.go |
|
||||
| JetStream | jetstream.go, stream.go, consumer.go, filestore.go, memstore.go, raft.go |
|
||||
| WebSocket | websocket.go |
|
||||
| MQTT | mqtt.go |
|
||||
| TLS | TLS config in opts.go, various TLS setup in server.go |
|
||||
Reference in New Issue
Block a user