feat: add structured logging, Shouldly assertions, CPM, and project documentation
- Add Microsoft.Extensions.Logging + Serilog to NatsServer and NatsClient - Convert all test assertions from xUnit Assert to Shouldly - Add NSubstitute package for future mocking needs - Introduce Central Package Management via Directory.Packages.props - Add documentation_rules.md with style guide, generation/update rules, component map - Generate 10 documentation files across 5 component folders (GettingStarted, Protocol, Subscriptions, Server, Configuration/Operations) - Update CLAUDE.md with logging, testing, porting, agent model, CPM, and documentation guidance
This commit is contained in:
Joseph Doherty
223
Documentation/Server/Overview.md
Normal file
223
Documentation/Server/Overview.md
Normal file
@@ -0,0 +1,223 @@
|
||||
# Server Overview
|
||||
|
||||
`NatsServer` is the top-level orchestrator: it binds the TCP listener, accepts incoming connections, and routes published messages to matching subscribers. Each connected client is managed by a `NatsClient` instance; `NatsServer` coordinates them through two interfaces that decouple message routing from connection management.
|
||||
|
||||
## Key Concepts
|
||||
|
||||
### Interfaces
|
||||
|
||||
`NatsServer` exposes two interfaces that `NatsClient` depends on:
|
||||
|
||||
```csharp
|
||||
public interface IMessageRouter
|
||||
{
|
||||
void ProcessMessage(string subject, string? replyTo, ReadOnlyMemory<byte> headers,
|
||||
ReadOnlyMemory<byte> payload, NatsClient sender);
|
||||
void RemoveClient(NatsClient client);
|
||||
}
|
||||
|
||||
public interface ISubListAccess
|
||||
{
|
||||
SubList SubList { get; }
|
||||
}
|
||||
```
|
||||
|
||||
`IMessageRouter` is the surface `NatsClient` calls when a PUB command arrives. `ISubListAccess` gives `NatsClient` access to the shared `SubList` so it can insert and remove subscriptions directly — without needing a concrete reference to `NatsServer`. Both interfaces are implemented by `NatsServer`, and both are injected into `NatsClient` through the `Router` property after construction.
|
||||
|
||||
Defining them separately makes unit testing straightforward: a test can supply a stub `IMessageRouter` without standing up a real server.
|
||||
|
||||
### Fields and State
|
||||
|
||||
```csharp
|
||||
public sealed class NatsServer : IMessageRouter, ISubListAccess, IDisposable
|
||||
{
|
||||
private readonly NatsOptions _options;
|
||||
private readonly ConcurrentDictionary<ulong, NatsClient> _clients = new();
|
||||
private readonly SubList _subList = new();
|
||||
private readonly ServerInfo _serverInfo;
|
||||
private readonly ILogger<NatsServer> _logger;
|
||||
private readonly ILoggerFactory _loggerFactory;
|
||||
private Socket? _listener;
|
||||
private ulong _nextClientId;
|
||||
|
||||
public SubList SubList => _subList;
|
||||
}
|
||||
```
|
||||
|
||||
`_clients` tracks every live connection. `_nextClientId` is incremented with `Interlocked.Increment` for each accepted socket, producing monotonically increasing client IDs without a lock. `_loggerFactory` is retained so per-client loggers can be created at accept time, each tagged with the client ID.
|
||||
|
||||
### Constructor
|
||||
|
||||
The constructor takes `NatsOptions` and `ILoggerFactory`. It builds a `ServerInfo` struct that is sent to every connecting client in the initial INFO message:
|
||||
|
||||
```csharp
|
||||
public NatsServer(NatsOptions options, ILoggerFactory loggerFactory)
|
||||
{
|
||||
_options = options;
|
||||
_loggerFactory = loggerFactory;
|
||||
_logger = loggerFactory.CreateLogger<NatsServer>();
|
||||
_serverInfo = new ServerInfo
|
||||
{
|
||||
ServerId = Guid.NewGuid().ToString("N")[..20].ToUpperInvariant(),
|
||||
ServerName = options.ServerName ?? $"nats-dotnet-{Environment.MachineName}",
|
||||
Version = NatsProtocol.Version,
|
||||
Host = options.Host,
|
||||
Port = options.Port,
|
||||
MaxPayload = options.MaxPayload,
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
The `ServerId` is derived from a GUID — taking the first 20 characters of its `"N"` format (32 hex digits, no hyphens) and uppercasing them. This matches the fixed-length alphanumeric server ID format used by the Go server.
|
||||
|
||||
## Accept Loop
|
||||
|
||||
`StartAsync` binds the socket, enables `SO_REUSEADDR` so the port can be reused immediately after a restart, and enters an async accept loop:
|
||||
|
||||
```csharp
|
||||
public async Task StartAsync(CancellationToken ct)
|
||||
{
|
||||
_listener = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
|
||||
_listener.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, true);
|
||||
_listener.Bind(new IPEndPoint(
|
||||
_options.Host == "0.0.0.0" ? IPAddress.Any : IPAddress.Parse(_options.Host),
|
||||
_options.Port));
|
||||
_listener.Listen(128);
|
||||
|
||||
while (!ct.IsCancellationRequested)
|
||||
{
|
||||
var socket = await _listener.AcceptAsync(ct);
|
||||
var clientId = Interlocked.Increment(ref _nextClientId);
|
||||
|
||||
var clientLogger = _loggerFactory.CreateLogger($"NATS.Server.NatsClient[{clientId}]");
|
||||
var client = new NatsClient(clientId, socket, _options, _serverInfo, clientLogger);
|
||||
client.Router = this;
|
||||
_clients[clientId] = client;
|
||||
|
||||
_ = RunClientAsync(client, ct);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`RunClientAsync` is fire-and-forget (`_ = ...`). The accept loop does not await it, so accepting new connections is not blocked by any single client's I/O. Each client runs concurrently on the thread pool.
|
||||
|
||||
The backlog of 128 passed to `Listen` controls the OS-level queue of unaccepted connections — matching the Go server default.
|
||||
|
||||
## Message Routing
|
||||
|
||||
`ProcessMessage` is called by `NatsClient` for every PUB or HPUB command. It is the hot path: called once per published message.
|
||||
|
||||
```csharp
|
||||
public void ProcessMessage(string subject, string? replyTo, ReadOnlyMemory<byte> headers,
|
||||
ReadOnlyMemory<byte> payload, NatsClient sender)
|
||||
{
|
||||
var result = _subList.Match(subject);
|
||||
|
||||
// Deliver to plain subscribers
|
||||
foreach (var sub in result.PlainSubs)
|
||||
{
|
||||
if (sub.Client == null || sub.Client == sender && !(sender.ClientOpts?.Echo ?? true))
|
||||
continue;
|
||||
|
||||
DeliverMessage(sub, subject, replyTo, headers, payload);
|
||||
}
|
||||
|
||||
// Deliver to one member of each queue group (round-robin)
|
||||
foreach (var queueGroup in result.QueueSubs)
|
||||
{
|
||||
if (queueGroup.Length == 0) continue;
|
||||
|
||||
var idx = Math.Abs((int)Interlocked.Increment(ref sender.OutMsgs)) % queueGroup.Length;
|
||||
Interlocked.Decrement(ref sender.OutMsgs);
|
||||
|
||||
for (int attempt = 0; attempt < queueGroup.Length; attempt++)
|
||||
{
|
||||
var sub = queueGroup[(idx + attempt) % queueGroup.Length];
|
||||
if (sub.Client != null && (sub.Client != sender || (sender.ClientOpts?.Echo ?? true)))
|
||||
{
|
||||
DeliverMessage(sub, subject, replyTo, headers, payload);
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Plain subscriber delivery
|
||||
|
||||
Each subscription in `result.PlainSubs` receives the message unless:
|
||||
|
||||
- `sub.Client` is null (the subscription was removed concurrently), or
|
||||
- the subscriber is the sender and the sender has `echo: false` in its CONNECT options.
|
||||
|
||||
The `echo` flag defaults to `true` in `ClientOptions`, so publishers receive their own messages unless they explicitly opt out.
|
||||
|
||||
### Queue group delivery
|
||||
|
||||
Queue groups provide load-balanced fan-out: exactly one member of each group receives each message. The selection uses a round-robin counter derived from `sender.OutMsgs`. An `Interlocked.Increment` picks the starting index; the `Interlocked.Decrement` immediately after undoes the side effect on the stat, since `OutMsgs` will be incremented correctly inside `SendMessageAsync` when the message is actually dispatched.
|
||||
|
||||
The loop walks from the selected index, wrapping around, until it finds an eligible member (non-null client, echo check). This handles stale subscriptions where the client has disconnected but the subscription object has not yet been cleaned up.
|
||||
|
||||
### DeliverMessage and auto-unsub
|
||||
|
||||
```csharp
|
||||
private static void DeliverMessage(Subscription sub, string subject, string? replyTo,
|
||||
ReadOnlyMemory<byte> headers, ReadOnlyMemory<byte> payload)
|
||||
{
|
||||
var client = sub.Client;
|
||||
if (client == null) return;
|
||||
|
||||
var count = Interlocked.Increment(ref sub.MessageCount);
|
||||
if (sub.MaxMessages > 0 && count > sub.MaxMessages)
|
||||
return;
|
||||
|
||||
_ = client.SendMessageAsync(subject, sub.Sid, replyTo, headers, payload, CancellationToken.None);
|
||||
}
|
||||
```
|
||||
|
||||
`MessageCount` is incremented atomically before the send. If it exceeds `MaxMessages` (set by an UNSUB with a message count argument), the message is silently dropped. The subscription itself is not removed here — removal happens when the client processes the count limit through `ProcessUnsub`, or when the client disconnects and `RemoveAllSubscriptions` is called.
|
||||
|
||||
`SendMessageAsync` is again fire-and-forget. Multiple deliveries to different clients happen concurrently.
|
||||
|
||||
## Client Removal
|
||||
|
||||
```csharp
|
||||
public void RemoveClient(NatsClient client)
|
||||
{
|
||||
_clients.TryRemove(client.Id, out _);
|
||||
client.RemoveAllSubscriptions(_subList);
|
||||
}
|
||||
```
|
||||
|
||||
`RemoveClient` is called either from `RunClientAsync`'s `finally` block (after a client disconnects or errors) or from `NatsClient.RunAsync`'s own `finally` block. Both paths may call it; `TryRemove` is idempotent, so double-calls are safe. After removal from `_clients`, all subscriptions belonging to that client are purged from the `SubList` trie and its internal cache.
|
||||
|
||||
## Shutdown and Dispose
|
||||
|
||||
```csharp
|
||||
public void Dispose()
|
||||
{
|
||||
_listener?.Dispose();
|
||||
foreach (var client in _clients.Values)
|
||||
client.Dispose();
|
||||
_subList.Dispose();
|
||||
}
|
||||
```
|
||||
|
||||
Disposing the listener socket causes `AcceptAsync` to throw, which unwinds `StartAsync`. Client sockets are disposed, which closes their `NetworkStream` and causes their read loops to terminate. `SubList.Dispose` releases its `ReaderWriterLockSlim`.
|
||||
|
||||
## Go Reference
|
||||
|
||||
The Go counterpart is `golang/nats-server/server/server.go`. Key differences in the .NET port:
|
||||
|
||||
- Go uses goroutines for the accept loop and per-client read/write loops; the .NET port uses `async`/`await` with `Task`.
|
||||
- Go uses `sync/atomic` for client ID generation; the .NET port uses `Interlocked.Increment`.
|
||||
- Go passes the server to clients via the `srv` field on the client struct; the .NET port uses the `IMessageRouter` interface through the `Router` property.
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [Client Connection Handler](Client.md)
|
||||
- [SubList Trie](../Subscriptions/SubList.md)
|
||||
- [Protocol Overview](../Protocol/Overview.md)
|
||||
- [Configuration](../Configuration/Overview.md)
|
||||
|
||||
<!-- Last verified against codebase: 2026-02-22 -->
|
||||
Reference in New Issue
Block a user