From 0ea71ace79e77e6d0ba232898f6c3e2fb70df64c Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Sun, 22 Feb 2026 19:37:32 -0500 Subject: [PATCH] Add CLAUDE.md and base server design document Design covers the minimal NATS server port: pub/sub with wildcards and queue groups over System.IO.Pipelines, targeting .NET 10. --- CLAUDE.md | 116 ++++++++++++++++ docs/plans/2026-02-22-base-server-design.md | 140 ++++++++++++++++++++ 2 files changed, 256 insertions(+) create mode 100644 CLAUDE.md create mode 100644 docs/plans/2026-02-22-base-server-design.md diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..556142d --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,116 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## Project Overview + +This project ports the [NATS server](https://github.com/nats-io/nats-server) from Go to .NET 10 / C#. The Go reference implementation lives in `golang/nats-server/`. The .NET port lives at the repository root. + +NATS is a high-performance publish-subscribe messaging system. It supports wildcards (`*` single token, `>` multi-token), queue groups for load balancing, request-reply, clustering (full-mesh routes, gateways, leaf nodes), and persistent streaming via JetStream. + +## Build & Test Commands + +```bash +# Build the solution +dotnet build + +# Run all tests +dotnet test + +# Run a single test project +dotnet test + +# Run a specific test by name +dotnet test --filter "FullyQualifiedName~TestClassName.TestMethodName" + +# Run tests with verbose output +dotnet test -v normal + +# Clean and rebuild +dotnet clean && dotnet build +``` + +## Go Reference Commands + +```bash +# Build the Go reference server +cd golang/nats-server && go build + +# Run Go tests for a specific area +cd golang/nats-server && go test -v -run TestName ./server/ -count=1 -timeout=30m + +# Run all Go server tests (slow, ~30min) +cd golang/nats-server && go test -v ./server/ -count=1 -timeout=30m +``` + +## Architecture: NATS Server (Reference) + +The Go source in `golang/nats-server/server/` is the authoritative reference. Key files by subsystem: + +### Core Message Path +- **`server.go`** — Server struct, startup lifecycle (`NewServer` → `Run` → `WaitForShutdown`), listener management +- **`client.go`** (6700 lines) — Connection handling, `readLoop`/`writeLoop` goroutines, per-client subscription tracking, dynamic buffer sizing (512→65536 bytes), client types: `CLIENT`, `ROUTER`, `GATEWAY`, `LEAF`, `SYSTEM` +- **`parser.go`** — Protocol state machine. Text protocol: `PUB`, `SUB`, `UNSUB`, `CONNECT`, `INFO`, `PING/PONG`, `MSG`. Extended: `HPUB/HMSG` (headers), `RPUB/RMSG` (routes). Control line limit: 4096 bytes. Default max payload: 1MB. +- **`sublist.go`** — Trie-based subject matcher with wildcard support. Nodes have `psubs` (plain), `qsubs` (queue groups), special pointers for `*` and `>` wildcards. Results are cached with atomic generation IDs for invalidation. + +### Authentication & Accounts +- **`auth.go`** — Auth mechanisms: username/password, token, NKeys (Ed25519), JWT, external auth callout, LDAP +- **`accounts.go`** (137KB) — Multi-tenant account isolation. Each account has its own `Sublist`, client set, and subject namespace. Supports exports/imports between accounts, service latency tracking. +- **`jwt.go`**, **`nkey.go`** — JWT claims parsing and NKey validation + +### Clustering +- **`route.go`** — Full-mesh cluster routes. Route pooling (default 3 connections per peer). Account-specific dedicated routes. Protocol: `RS+`/`RS-` for subscribe propagation, `RMSG` for routed messages. +- **`gateway.go`** (103KB) — Inter-cluster bridges. Interest-only mode optimizes traffic. Reply subject mapping (`_GR_.` prefix) avoids cross-cluster conflicts. +- **`leafnode.go`** — Hub-and-spoke topology for edge deployments. Only subscribed subjects shared with hub. Loop detection via `$LDS.` prefix. + +### JetStream (Persistence) +- **`jetstream.go`** — Orchestration, API subject handlers (`$JS.API.*`) +- **`stream.go`** (8000 lines) — Stream lifecycle, retention policies (Limits, Interest, WorkQueue), subject transforms, mirroring/sourcing +- **`consumer.go`** — Stateful readers. Push vs pull delivery. Ack policies: None, All, Explicit. Redelivery tracking, priority groups. +- **`filestore.go`** (337KB) — Block-based persistent storage with S2 compression, encryption (ChaCha20/AES-GCM), indexing +- **`memstore.go`** — In-memory storage with hash-wheel TTL expiration +- **`raft.go`** — RAFT consensus for clustered JetStream. Meta-cluster for metadata, per-stream/consumer RAFT groups. + +### Configuration & Monitoring +- **`opts.go`** — CLI flags + config file loading. CLI overrides config. Supports hot reload on signal. +- **`monitor.go`** — HTTP endpoints: `/varz`, `/connz`, `/routez`, `/gatewayz`, `/jsz`, `/healthz` +- **`conf/`** — Config file parser (custom format with includes) + +### Internal Data Structures +- **`server/avl/`** — AVL tree for sparse sequence sets (ack tracking) +- **`server/stree/`** — Subject tree for per-subject state in streams +- **`server/gsl/`** — Generic subject list, optimized trie +- **`server/thw/`** — Time hash wheel for efficient TTL expiration + +## Key Porting Considerations + +**Concurrency model:** Go uses goroutines (one per connection readLoop + writeLoop). Map to async/await with `Task`-based I/O. Use `Channel` or `Pipe` for producer-consumer patterns where Go uses channels. + +**Locking:** Go `sync.RWMutex` maps to `ReaderWriterLockSlim`. Go `sync.Map` maps to `ConcurrentDictionary`. Go `atomic` operations map to `Interlocked` or `volatile`. + +**Subject matching:** The `Sublist` trie is performance-critical. Every published message triggers a `Match()` call. Cache invalidation uses atomic generation counters. + +**Protocol parsing:** The parser is a byte-by-byte state machine. In .NET, use `System.IO.Pipelines` for zero-copy parsing with `ReadOnlySequence`. + +**Buffer management:** Go uses `[]byte` slices with pooling. Map to `ArrayPool` and `Memory`/`Span`. + +**Compression:** NATS uses S2 (Snappy variant) for route/gateway compression. Use an equivalent .NET S2 library or IronSnappy. + +**Ports:** Client=4222, Cluster=6222, Monitoring=8222, Leaf=5222, Gateway=7222. + +## Message Flow Summary + +``` +Client PUB → parser → permission check → Sublist.Match() → + ├─ Local subscribers: MSG to each (queue subs: pick one per group) + ├─ Cluster routes: RMSG to peers (who deliver to their locals) + ├─ Gateways: forward to interested remote clusters + └─ JetStream: if subject matches a stream, store + deliver to consumers +``` + +## Conventions + +- Reference the Go implementation file and line when porting a subsystem +- Maintain protocol compatibility — the .NET server must interoperate with existing NATS clients and Go servers in a cluster +- Use the same configuration file format as the Go server (parsed by `conf/` package) +- Match the Go server's monitoring JSON response shapes for tooling compatibility diff --git a/docs/plans/2026-02-22-base-server-design.md b/docs/plans/2026-02-22-base-server-design.md new file mode 100644 index 0000000..9f1326c --- /dev/null +++ b/docs/plans/2026-02-22-base-server-design.md @@ -0,0 +1,140 @@ +# NATS .NET Base Server Design + +**Date:** 2026-02-22 +**Scope:** Minimal single-node NATS server — pub/sub, queue groups, wildcards. No auth, clustering, JetStream, or monitoring. +**Target:** .NET 10, C# +**Approach:** Bottom-up (Sublist → Parser → Client → Server) + +## Decisions + +- **Framework:** .NET 10 +- **I/O:** System.IO.Pipelines for zero-copy protocol parsing +- **Structure:** Library (NATS.Server) + Host (NATS.Server.Host) + Tests (NATS.Server.Tests) +- **Test framework:** xUnit +- **Queue group selection:** Round-robin with atomic counter + +## Solution Layout + +``` +natsdotnet/ +├── NatsDotNet.sln +├── src/ +│ ├── NATS.Server/ # Core library +│ │ ├── NatsServer.cs # Server orchestrator +│ │ ├── NatsClient.cs # Client connection +│ │ ├── NatsOptions.cs # Configuration +│ │ ├── Protocol/ +│ │ │ ├── NatsProtocol.cs # Constants, op codes +│ │ │ └── NatsParser.cs # Pipeline-based parser +│ │ └── Subscriptions/ +│ │ ├── Subscription.cs +│ │ ├── SubjectMatch.cs +│ │ └── SubList.cs # Trie + cache +│ └── NATS.Server.Host/ # Console app +│ └── Program.cs +└── tests/ + └── NATS.Server.Tests/ +``` + +## Component Designs + +### 1. SubList (subject matching trie) + +Reference: `golang/nats-server/server/sublist.go` + +**Structures:** +- `SubList` — root trie + cache (`Dictionary`) + `ReaderWriterLockSlim` + atomic `genId` +- `TrieLevel` — `Dictionary` for literal tokens, nullable `pwc` (`*`) and `fwc` (`>`) pointers +- `TrieNode` — optional `next: TrieLevel`, `HashSet` for plain subs, `Dictionary>` for queue subs +- `Subscription` — client ref, subject, queue, sid, atomic message count, max messages +- `SubListResult` — `Subscription[]` plain subs, `Subscription[][]` queue subs (grouped) + +**Operations:** +- `Insert(Subscription)` — walk trie by `.`-split tokens, create nodes, add sub, increment genId (clears cache) +- `Remove(Subscription)` — reverse, prune empty nodes, increment genId +- `Match(ReadOnlySpan)` — cache check (convert to string key on miss), walk trie collecting matches from literal + `*` + `>` nodes +- Thread safety: read lock for Match, write lock for Insert/Remove + +### 2. Protocol Parser + +Reference: `golang/nats-server/server/parser.go` + +Operates on `PipeReader`. Scans for `\r\n` to delimit control lines. Identifies command by first bytes. For `PUB`/`HPUB`, reads control line then exactly `size` bytes of payload + `\r\n`. + +**Commands (base server):** + +| Command | Direction | Control line format | +|---------|-----------|-------------------| +| INFO | S→C | `INFO {json}\r\n` | +| CONNECT | C→S | `CONNECT {json}\r\n` | +| PUB | C→S | `PUB subject [reply] size\r\n[payload]\r\n` | +| HPUB | C→S | `HPUB subject [reply] hdr_size total_size\r\n[hdrs+payload]\r\n` | +| SUB | C→S | `SUB subject [queue] sid\r\n` | +| UNSUB | C→S | `UNSUB sid [max_msgs]\r\n` | +| MSG | S→C | `MSG subject sid [reply] size\r\n[payload]\r\n` | +| HMSG | S→C | `HMSG subject sid [reply] hdr_size total_size\r\n[hdrs+payload]\r\n` | +| PING/PONG | Both | `PING\r\n` / `PONG\r\n` | +| +OK / -ERR | S→C | `+OK\r\n` / `-ERR 'msg'\r\n` | + +**Design choices:** +- Parser returns parsed command structs, does not dispatch +- No string allocation on hot path — subject matching uses `Span` +- Max control line: 4096 bytes, max payload: 1MB default (configurable) + +### 3. NatsClient (connection handler) + +Reference: `golang/nats-server/server/client.go` + +**Fields:** unique id, Socket, IDuplexPipe, back-ref to server, subs dict (sid→Subscription), ClientOptions (from CONNECT json), stats (Interlocked counters), SemaphoreSlim for write serialization. + +**Lifecycle:** +1. Server accepts socket, creates NatsClient +2. `RunAsync(CancellationToken)`: send INFO → read loop (parse + dispatch commands) +3. Dispatch: CONNECT→validate, SUB→insert into SubList, UNSUB→remove, PUB→`server.ProcessMessage()`, PING→PONG +4. On disconnect: remove all subs, remove from server clients dict + +**Writing:** `SendMessageAsync()` formats MSG/HMSG, acquires write lock, writes to PipeWriter, flushes. + +### 4. NatsServer (orchestrator) + +Reference: `golang/nats-server/server/server.go` + +**Fields:** NatsOptions, TCP listener Socket, `ConcurrentDictionary`, SubList, atomic next client ID, ServerInfo struct, CancellationTokenSource. + +**Core methods:** +- `StartAsync()` — bind, accept loop +- `ProcessMessage(subject, reply, headers, payload, sender)` — SubList.Match → deliver to plain subs + pick one per queue group (round-robin) +- `RemoveClient(client)` — cleanup subs and client dict +- `Shutdown()` — cancel, close listener, drain + +### 5. NatsOptions + +Reference: `golang/nats-server/server/opts.go` + +Minimal for base: Host, Port (default 4222), ServerName, MaxPayload (1MB), MaxControlLine (4096), MaxConnections, PingInterval, MaxPingsOut. + +## Testing Strategy + +**Unit tests:** +- SubList: insert/remove/match with literals, `*`, `>`, queue groups, cache invalidation. Port Go test cases. +- Parser: each command type, malformed input, partial reads, boundary sizes. +- Client protocol: CONNECT handshake, SUB/UNSUB lifecycle, PUB delivery, PING/PONG via loopback sockets. + +**Integration tests (using NATS.Client.Core NuGet):** +- Basic pub/sub, wildcard matching, queue group distribution, fan-out, UNSUB + auto-unsub, PING/PONG. + +## Module Boundaries (future) + +These are explicitly excluded from this design and will be added as separate modules later: + +| Module | Key files in Go reference | +|--------|--------------------------| +| Authentication | auth.go, auth_callout.go, nkey.go, jwt.go | +| Monitoring HTTP | monitor.go | +| Clustering/Routes | route.go | +| Gateways | gateway.go | +| Leaf Nodes | leafnode.go | +| JetStream | jetstream.go, stream.go, consumer.go, filestore.go, memstore.go, raft.go | +| WebSocket | websocket.go | +| MQTT | mqtt.go | +| TLS | TLS config in opts.go, various TLS setup in server.go |