dohertj2/natsdotnet
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: add structured logging, Shouldly assertions, CPM, and project documentation

Browse Source 
- Add Microsoft.Extensions.Logging + Serilog to NatsServer and NatsClient
- Convert all test assertions from xUnit Assert to Shouldly
- Add NSubstitute package for future mocking needs
- Introduce Central Package Management via Directory.Packages.props
- Add documentation_rules.md with style guide, generation/update rules, component map
- Generate 10 documentation files across 5 component folders (GettingStarted, Protocol, Subscriptions, Server, Configuration/Operations)
- Update CLAUDE.md with logging, testing, porting, agent model, CPM, and documentation guidance
This commit is contained in:
Joseph Doherty
2026-02-22 21:05:53 -05:00
parent b9f4dec523
commit 539b2b7588
25 changed files with 2734 additions and 110 deletions
Download Patch File Download Diff File Expand all files Collapse all files

169
Documentation/Subscriptions/Overview.md Normal file
View File

@@ -0,0 +1,169 @@
# Subscriptions Overview
The subscription system maps published subjects to interested subscribers. It consists of four classes: `Subscription` (the subscription model), `SubListResult` (a match result container), `SubjectMatch` (subject validation and wildcard matching), and `SubList` (the trie that routes messages to subscribers).
Go reference: `golang/nats-server/server/sublist.go`
---
## Subjects
A NATS subject is a dot-separated string of tokens: `foo.bar.baz`. Each token is a non-empty sequence of non-whitespace characters. The token separator is `.`.
### Wildcards
Two wildcard tokens are supported:
| Token | Name | Matches |
|-------|------|---------|
| `*` | Partial wildcard (Pwc) | Exactly one token at that position |
| `>` | Full wildcard (Fwc) | One or more remaining tokens |
Rules:
- `foo.*` matches `foo.bar` but not `foo.bar.baz` (only one token after `foo`).
- `foo.>` matches `foo.bar` and `foo.bar.baz` (one or more tokens after `foo`).
- `>` must be the last token in a subject. `foo.>.bar` is invalid.
- Publishers cannot use wildcards — only subscribers can.
### Queue Groups
Subscribers with the same queue name form a queue group. When a message matches a queue group, exactly one member of that group receives the message. The server selects a member per published message, which distributes load across the group.
Queue subscriptions are independent of plain subscriptions. If a subject has both plain subscribers and queue group subscribers, all plain subscribers receive the message and one member from each queue group receives it.
---
## The `Subscription` Class
A `Subscription` represents one subscriber's interest in a subject.
```csharp
public sealed class Subscription
{
public required string Subject { get; init; }
public string? Queue { get; init; }
public required string Sid { get; init; }
public long MessageCount; // Interlocked
public long MaxMessages; // 0 = unlimited
public NatsClient? Client { get; set; }
}
```
- `Subject` — the subject pattern, which may contain wildcards.
- `Queue` — the queue group name. `null` means this is a plain subscription.
- `Sid` — the subscription ID string assigned by the client. Used to identify the subscription in `UNSUB` commands.
- `MessageCount` — updated with `Interlocked.Increment` on each delivered message. Compared against `MaxMessages` to implement `UNSUB maxMessages`.
- `MaxMessages` — the auto-unsubscribe limit. `0` means unlimited.
- `Client` — back-reference to the `NatsClient` that owns this subscription. Used during message delivery to write `MSG` frames to the connection.
---
## The `SubListResult` Class
`SubListResult` is the return type of `SubList.Match()`. It separates plain subscribers from queue group subscribers.
```csharp
public sealed class SubListResult
{
public static readonly SubListResult Empty = new([], []);
public Subscription[] PlainSubs { get; }
public Subscription[][] QueueSubs { get; } // outer = groups, inner = members
public SubListResult(Subscription[] plainSubs, Subscription[][] queueSubs)
{
PlainSubs = plainSubs;
QueueSubs = queueSubs;
}
}
```
- `PlainSubs` — all plain subscribers whose subject pattern matches the published subject.
- `QueueSubs` — one entry per queue group. Each inner array contains all members of that group. The message router picks one member per group.
- `Empty` — a static singleton returned when no subscriptions match. Avoids allocation on the common case of no interest.
---
## The `SubjectMatch` Class
`SubjectMatch` is a static utility class for subject validation and wildcard pattern matching. It defines the three fundamental constants used throughout the subscription system:
```csharp
public static class SubjectMatch
{
public const char Pwc = '*'; // partial wildcard
public const char Fwc = '>'; // full wildcard
public const char Sep = '.'; // token separator
...
}
```
### Validation methods
**`IsValidSubject(string subject)`** — returns `true` if the subject is a well-formed NATS subject. Rejects:
- Empty strings.
- Empty tokens (consecutive separators, or leading/trailing separators).
- Tokens containing whitespace characters.
- Any token after `>` (full wildcard must be last).
**`IsLiteral(string subject)`** — returns `true` if the subject contains no wildcards. A character is treated as a wildcard only when it appears as a complete token (preceded and followed by `.` or the string boundary).
**`IsValidPublishSubject(string subject)`** — returns `true` if the subject is both valid and literal. Publishers must use literal subjects; this is the combined check applied before inserting a message into the routing path.
### Pattern matching
**`MatchLiteral(string literal, string pattern)`** — tests whether a literal subject matches a pattern that may contain wildcards. Used by the cache invalidation logic to determine which cached results are affected when a wildcard subscription is added or removed.
```csharp
public static bool MatchLiteral(string literal, string pattern)
{
int li = 0, pi = 0;
while (pi < pattern.Length)
{
// Get next pattern token
int pTokenStart = pi;
while (pi < pattern.Length && pattern[pi] != Sep)
pi++;
int pTokenLen = pi - pTokenStart;
if (pi < pattern.Length)
pi++; // skip separator
// Full wildcard -- matches everything remaining
if (pTokenLen == 1 && pattern[pTokenStart] == Fwc)
return li < literal.Length; // must have at least one token left
// Get next literal token
if (li >= literal.Length)
return false;
int lTokenStart = li;
while (li < literal.Length && literal[li] != Sep)
li++;
int lTokenLen = li - lTokenStart;
if (li < literal.Length)
li++; // skip separator
// Partial wildcard -- matches any single token
if (pTokenLen == 1 && pattern[pTokenStart] == Pwc)
continue;
// Literal comparison
if (pTokenLen != lTokenLen)
return false;
if (string.Compare(literal, lTokenStart, pattern, pTokenStart, pTokenLen, StringComparison.Ordinal) != 0)
return false;
}
return li >= literal.Length; // both exhausted
}
```
The algorithm walks both strings token by token without allocating. `>` short-circuits immediately, returning `true` as long as at least one literal token remains. `*` skips the corresponding literal token without comparison. Literal tokens are compared with `StringComparison.Ordinal`.
---
## Related Documentation
- [SubList Trie](../Subscriptions/SubList.md)
<!-- Last verified against codebase: 2026-02-22 -->

242
Documentation/Subscriptions/SubList.md Normal file
View File

@@ -0,0 +1,242 @@
# SubList
`SubList` is the subscription routing trie. Every published message triggers a `Match()` call to find all interested subscribers. `SubList` stores subscriptions indexed by their subject tokens and returns a `SubListResult` containing both plain subscribers and queue groups.
Go reference: `golang/nats-server/server/sublist.go`
---
## Thread Safety
`SubList` uses a `ReaderWriterLockSlim` (`_lock`) with the following locking discipline:
| Operation | Lock |
|-----------|------|
| `Count` read | Read lock |
| `Match()` — cache hit | Read lock only |
| `Match()` — cache miss | Write lock (to update cache) |
| `Insert()` | Write lock |
| `Remove()` | Write lock |
Cache misses in `Match()` require a write lock because the cache must be updated after the trie traversal. To avoid a race between the read-lock check and the write-lock update, `Match()` uses double-checked locking: after acquiring the write lock, it checks the cache again before doing trie work.
---
## Trie Structure
The trie is built from two private classes, `TrieLevel` and `TrieNode`, nested inside `SubList`.
### `TrieLevel` and `TrieNode`
```csharp
private sealed class TrieLevel
{
public readonly Dictionary<string, TrieNode> Nodes = new(StringComparer.Ordinal);
public TrieNode? Pwc; // partial wildcard (*)
public TrieNode? Fwc; // full wildcard (>)
}
private sealed class TrieNode
{
public TrieLevel? Next;
public readonly HashSet<Subscription> PlainSubs = [];
public readonly Dictionary<string, HashSet<Subscription>> QueueSubs = new(StringComparer.Ordinal);
public bool IsEmpty => PlainSubs.Count == 0 && QueueSubs.Count == 0 &&
(Next == null || (Next.Nodes.Count == 0 && Next.Pwc == null && Next.Fwc == null));
}
```
Each level in the trie represents one token position in a subject. A `TrieLevel` holds:
- `Nodes` — a dictionary keyed by literal token string, mapping to the `TrieNode` for that token. Uses `StringComparer.Ordinal` for performance.
- `Pwc` — the node for the `*` wildcard at this level, or `null` if no `*` subscriptions exist at this depth.
- `Fwc` — the node for the `>` wildcard at this level, or `null` if no `>` subscriptions exist at this depth.
A `TrieNode` sits at the boundary between two levels. It holds the subscriptions registered for subjects whose last token leads to this node:
- `PlainSubs` — a `HashSet<Subscription>` of plain (non-queue) subscribers.
- `QueueSubs` — a dictionary from queue name to the set of members in that queue group. Uses `StringComparer.Ordinal`.
- `Next` — the next `TrieLevel` for deeper token positions. `null` for leaf nodes.
- `IsEmpty` — `true` when the node and all its descendants have no subscriptions. Used during `Remove()` to prune dead branches.
The trie root is a `TrieLevel` (`_root`) with no parent node.
### `TokenEnumerator`
`TokenEnumerator` is a `ref struct` that splits a subject string by `.` without allocating. It operates on a `ReadOnlySpan<char>` derived from the original string.
```csharp
private ref struct TokenEnumerator
{
private ReadOnlySpan<char> _remaining;
public TokenEnumerator(string subject)
{
_remaining = subject.AsSpan();
Current = default;
}
public ReadOnlySpan<char> Current { get; private set; }
public TokenEnumerator GetEnumerator() => this;
public bool MoveNext()
{
if (_remaining.IsEmpty)
return false;
int sep = _remaining.IndexOf(SubjectMatch.Sep);
if (sep < 0)
{
Current = _remaining;
_remaining = default;
}
else
{
Current = _remaining[..sep];
_remaining = _remaining[(sep + 1)..];
}
return true;
}
}
```
`TokenEnumerator` implements the `foreach` pattern directly (via `GetEnumerator()` returning `this`), so it can be used in `foreach` loops without boxing. `Insert()` uses it during trie traversal to avoid string allocations per token.
---
## Insert
`Insert(Subscription sub)` adds a subscription to the trie under a write lock.
The method walks the trie one token at a time using `TokenEnumerator`. For each token:
- If the token is `*`, it creates or follows `level.Pwc`.
- If the token is `>`, it creates or follows `level.Fwc` and sets `sawFwc = true` to reject further tokens.
- Otherwise it creates or follows `level.Nodes[token]`.
At each step, `node.Next` is created if absent, and `level` advances to `node.Next`.
After all tokens are consumed, the subscription is added to the terminal node:
- Plain subscription: `node.PlainSubs.Add(sub)`.
- Queue subscription: `node.QueueSubs[sub.Queue].Add(sub)`, creating the inner `HashSet<Subscription>` if this is the first member of that group.
`_count` is incremented and `AddToCache` is called to update any cached results that would now include this subscription.
---
## Remove
`Remove(Subscription sub)` removes a subscription from the trie under a write lock.
The method walks the trie along the subscription's subject, recording the path as a `List<(TrieLevel, TrieNode, string token, bool isPwc, bool isFwc)>`. If any node along the path is missing, the method returns without error (the subscription was never inserted).
After locating the terminal node, the subscription is removed from `PlainSubs` or from the appropriate `QueueSubs` group. If the queue group becomes empty, its entry is removed from the dictionary.
If removal succeeds:
- `_count` is decremented.
- `RemoveFromCache` is called to invalidate affected cache entries.
- The path list is walked backwards. At each step, if `node.IsEmpty` is `true`, the node is removed from its parent level (`Pwc = null`, `Fwc = null`, or `Nodes.Remove(token)`). This prunes dead branches so the trie does not accumulate empty nodes over time.
---
## Match
`Match(string subject)` is called for every published message. It returns a `SubListResult` containing all matching plain and queue subscriptions.
### Cache check and fallback
```csharp
public SubListResult Match(string subject)
{
// Check cache under read lock first.
_lock.EnterReadLock();
try
{
if (_cache != null && _cache.TryGetValue(subject, out var cached))
return cached;
}
finally
{
_lock.ExitReadLock();
}
// Cache miss -- tokenize and match under write lock (needed for cache update).
var tokens = Tokenize(subject);
if (tokens == null)
return SubListResult.Empty;
_lock.EnterWriteLock();
try
{
// Re-check cache after acquiring write lock.
if (_cache != null && _cache.TryGetValue(subject, out var cached))
return cached;
var plainSubs = new List<Subscription>();
var queueSubs = new List<List<Subscription>>();
MatchLevel(_root, tokens, 0, plainSubs, queueSubs);
...
if (_cache != null)
{
_cache[subject] = result;
if (_cache.Count > CacheMax) { /* sweep */ }
}
return result;
}
finally { _lock.ExitWriteLock(); }
}
```
On a read-lock cache hit, `Match()` returns immediately with no trie traversal. On a miss, `Tokenize()` splits the subject before acquiring the write lock (subjects with empty tokens return `SubListResult.Empty` immediately). The write lock is then taken and the cache is checked again before invoking `MatchLevel`.
### `MatchLevel` traversal
`MatchLevel` is a recursive method that descends the trie matching tokens against the subject array. At each level, for each remaining token position:
1. If `level.Fwc` is set, all subscriptions from that node are added to the result. The `>` wildcard matches all remaining tokens, so no further recursion is needed for this branch.
2. If `level.Pwc` is set, `MatchLevel` recurses with the next token index and `pwc.Next` as the new level. This handles `*` matching the current token.
3. A literal dictionary lookup on `level.Nodes[tokens[i]]` advances the level pointer for the next iteration.
After all tokens are consumed, subscriptions from the final literal node and the final `*` position (if present at the last level) are added to the result. The `*` case at the last token requires explicit handling because the loop exits before the recursive call for `*` can execute.
`AddNodeToResults` flattens a node's `PlainSubs` into the accumulator list and merges its `QueueSubs` groups into the queue accumulator, combining groups by name across multiple matching nodes.
---
## Cache Strategy
The cache is a `Dictionary<string, SubListResult>` keyed by the literal published subject. All operations use `StringComparer.Ordinal`.
**Size limits:** The cache holds at most `CacheMax` (1024) entries. When `_cache.Count` exceeds this, a sweep removes entries until the count reaches `CacheSweep` (256). The sweep takes the first `count - 256` keys from the dictionary — no LRU ordering is maintained.
**`AddToCache`** is called from `Insert()` to keep cached results consistent after adding a subscription:
- For a literal subscription subject, `AddToCache` does a direct lookup. If the exact key is in the cache, it creates a new `SubListResult` with the subscription appended and replaces the cached entry.
- For a wildcard subscription subject, `AddToCache` scans all cached keys and updates any entry whose key is matched by `SubjectMatch.MatchLiteral(key, subject)`.
**`RemoveFromCache`** is called from `Remove()` to invalidate cached results after removing a subscription:
- For a literal subscription subject, `RemoveFromCache` removes the exact cache key.
- For a wildcard subscription subject, `RemoveFromCache` removes all cached keys matched by the pattern. Because it is difficult to reconstruct the correct result without a full trie traversal, invalidation is preferred over update.
The asymmetry between `AddToCache` (updates in place) and `RemoveFromCache` (invalidates) avoids a second trie traversal on removal at the cost of a cache miss on the next `Match()` for those keys.
---
## Disposal
`SubList` implements `IDisposable`. `Dispose()` releases the `ReaderWriterLockSlim`:
```csharp
public void Dispose() => _lock.Dispose();
```
`SubList` instances are owned by `NatsServer` and disposed during server shutdown.
---
## Related Documentation
- [Subscriptions Overview](../Subscriptions/Overview.md)
<!-- Last verified against codebase: 2026-02-22 -->