# SubList `SubList` is the subscription routing trie. Every published message triggers a `Match()` call to find all interested subscribers. `SubList` stores subscriptions indexed by their subject tokens and returns a `SubListResult` containing both plain subscribers and queue groups. Go reference: `golang/nats-server/server/sublist.go` --- ## Thread Safety `SubList` uses a `ReaderWriterLockSlim` (`_lock`) with the following locking discipline: | Operation | Lock | |-----------|------| | `Count` read | Read lock | | `Match()` — cache hit | Read lock only | | `Match()` — cache miss | Write lock (to update cache) | | `Insert()` | Write lock | | `Remove()` | Write lock | Cache misses in `Match()` require a write lock because the cache must be updated after the trie traversal. To avoid a race between the read-lock check and the write-lock update, `Match()` uses double-checked locking: after acquiring the write lock, it checks the cache again before doing trie work. --- ## Trie Structure The trie is built from two private classes, `TrieLevel` and `TrieNode`, nested inside `SubList`. ### `TrieLevel` and `TrieNode` ```csharp private sealed class TrieLevel { public readonly Dictionary Nodes = new(StringComparer.Ordinal); public TrieNode? Pwc; // partial wildcard (*) public TrieNode? Fwc; // full wildcard (>) } private sealed class TrieNode { public TrieLevel? Next; public readonly HashSet PlainSubs = []; public readonly Dictionary> QueueSubs = new(StringComparer.Ordinal); public bool IsEmpty => PlainSubs.Count == 0 && QueueSubs.Count == 0 && (Next == null || (Next.Nodes.Count == 0 && Next.Pwc == null && Next.Fwc == null)); } ``` Each level in the trie represents one token position in a subject. A `TrieLevel` holds: - `Nodes` — a dictionary keyed by literal token string, mapping to the `TrieNode` for that token. Uses `StringComparer.Ordinal` for performance. - `Pwc` — the node for the `*` wildcard at this level, or `null` if no `*` subscriptions exist at this depth. - `Fwc` — the node for the `>` wildcard at this level, or `null` if no `>` subscriptions exist at this depth. A `TrieNode` sits at the boundary between two levels. It holds the subscriptions registered for subjects whose last token leads to this node: - `PlainSubs` — a `HashSet` of plain (non-queue) subscribers. - `QueueSubs` — a dictionary from queue name to the set of members in that queue group. Uses `StringComparer.Ordinal`. - `Next` — the next `TrieLevel` for deeper token positions. `null` for leaf nodes. - `IsEmpty` — `true` when the node and all its descendants have no subscriptions. Used during `Remove()` to prune dead branches. The trie root is a `TrieLevel` (`_root`) with no parent node. ### `TokenEnumerator` `TokenEnumerator` is a `ref struct` that splits a subject string by `.` without allocating. It operates on a `ReadOnlySpan` derived from the original string. ```csharp private ref struct TokenEnumerator { private ReadOnlySpan _remaining; public TokenEnumerator(string subject) { _remaining = subject.AsSpan(); Current = default; } public ReadOnlySpan Current { get; private set; } public TokenEnumerator GetEnumerator() => this; public bool MoveNext() { if (_remaining.IsEmpty) return false; int sep = _remaining.IndexOf(SubjectMatch.Sep); if (sep < 0) { Current = _remaining; _remaining = default; } else { Current = _remaining[..sep]; _remaining = _remaining[(sep + 1)..]; } return true; } } ``` `TokenEnumerator` implements the `foreach` pattern directly (via `GetEnumerator()` returning `this`), so it can be used in `foreach` loops without boxing. `Insert()` uses it during trie traversal to avoid string allocations per token. --- ## Insert `Insert(Subscription sub)` adds a subscription to the trie under a write lock. The method walks the trie one token at a time using `TokenEnumerator`. For each token: - If the token is `*`, it creates or follows `level.Pwc`. - If the token is `>`, it creates or follows `level.Fwc` and sets `sawFwc = true` to reject further tokens. - Otherwise it creates or follows `level.Nodes[token]`. At each step, `node.Next` is created if absent, and `level` advances to `node.Next`. After all tokens are consumed, the subscription is added to the terminal node: - Plain subscription: `node.PlainSubs.Add(sub)`. - Queue subscription: `node.QueueSubs[sub.Queue].Add(sub)`, creating the inner `HashSet` if this is the first member of that group. `_count` is incremented and `AddToCache` is called to update any cached results that would now include this subscription. --- ## Remove `Remove(Subscription sub)` removes a subscription from the trie under a write lock. The method walks the trie along the subscription's subject, recording the path as a `List<(TrieLevel, TrieNode, string token, bool isPwc, bool isFwc)>`. If any node along the path is missing, the method returns without error (the subscription was never inserted). After locating the terminal node, the subscription is removed from `PlainSubs` or from the appropriate `QueueSubs` group. If the queue group becomes empty, its entry is removed from the dictionary. If removal succeeds: - `_count` is decremented. - `RemoveFromCache` is called to invalidate affected cache entries. - The path list is walked backwards. At each step, if `node.IsEmpty` is `true`, the node is removed from its parent level (`Pwc = null`, `Fwc = null`, or `Nodes.Remove(token)`). This prunes dead branches so the trie does not accumulate empty nodes over time. --- ## Match `Match(string subject)` is called for every published message. It returns a `SubListResult` containing all matching plain and queue subscriptions. ### Cache check and fallback ```csharp public SubListResult Match(string subject) { // Check cache under read lock first. _lock.EnterReadLock(); try { if (_cache != null && _cache.TryGetValue(subject, out var cached)) return cached; } finally { _lock.ExitReadLock(); } // Cache miss -- tokenize and match under write lock (needed for cache update). var tokens = Tokenize(subject); if (tokens == null) return SubListResult.Empty; _lock.EnterWriteLock(); try { // Re-check cache after acquiring write lock. if (_cache != null && _cache.TryGetValue(subject, out var cached)) return cached; var plainSubs = new List(); var queueSubs = new List>(); MatchLevel(_root, tokens, 0, plainSubs, queueSubs); ... if (_cache != null) { _cache[subject] = result; if (_cache.Count > CacheMax) { /* sweep */ } } return result; } finally { _lock.ExitWriteLock(); } } ``` On a read-lock cache hit, `Match()` returns immediately with no trie traversal. On a miss, `Tokenize()` splits the subject before acquiring the write lock (subjects with empty tokens return `SubListResult.Empty` immediately). The write lock is then taken and the cache is checked again before invoking `MatchLevel`. ### `MatchLevel` traversal `MatchLevel` is a recursive method that descends the trie matching tokens against the subject array. At each level, for each remaining token position: 1. If `level.Fwc` is set, all subscriptions from that node are added to the result. The `>` wildcard matches all remaining tokens, so no further recursion is needed for this branch. 2. If `level.Pwc` is set, `MatchLevel` recurses with the next token index and `pwc.Next` as the new level. This handles `*` matching the current token. 3. A literal dictionary lookup on `level.Nodes[tokens[i]]` advances the level pointer for the next iteration. After all tokens are consumed, subscriptions from the final literal node and the final `*` position (if present at the last level) are added to the result. The `*` case at the last token requires explicit handling because the loop exits before the recursive call for `*` can execute. `AddNodeToResults` flattens a node's `PlainSubs` into the accumulator list and merges its `QueueSubs` groups into the queue accumulator, combining groups by name across multiple matching nodes. --- ## Cache Strategy The cache is a `Dictionary` keyed by the literal published subject. All operations use `StringComparer.Ordinal`. **Size limits:** The cache holds at most `CacheMax` (1024) entries. When `_cache.Count` exceeds this, a sweep removes entries until the count reaches `CacheSweep` (256). The sweep takes the first `count - 256` keys from the dictionary — no LRU ordering is maintained. **`AddToCache`** is called from `Insert()` to keep cached results consistent after adding a subscription: - For a literal subscription subject, `AddToCache` does a direct lookup. If the exact key is in the cache, it creates a new `SubListResult` with the subscription appended and replaces the cached entry. - For a wildcard subscription subject, `AddToCache` scans all cached keys and updates any entry whose key is matched by `SubjectMatch.MatchLiteral(key, subject)`. **`RemoveFromCache`** is called from `Remove()` to invalidate cached results after removing a subscription: - For a literal subscription subject, `RemoveFromCache` removes the exact cache key. - For a wildcard subscription subject, `RemoveFromCache` removes all cached keys matched by the pattern. Because it is difficult to reconstruct the correct result without a full trie traversal, invalidation is preferred over update. The asymmetry between `AddToCache` (updates in place) and `RemoveFromCache` (invalidates) avoids a second trie traversal on removal at the cost of a cache miss on the next `Match()` for those keys. --- ## Disposal `SubList` implements `IDisposable`. `Dispose()` releases the `ReaderWriterLockSlim`: ```csharp public void Dispose() => _lock.Dispose(); ``` `SubList` instances are owned by `NatsServer` and disposed during server shutdown. --- ## Related Documentation - [Subscriptions Overview](../Subscriptions/Overview.md)