Improve source XML docs and refresh profiling artifacts

This captures the iterative CommentChecker cleanup plus updated snapshot/report outputs used to validate and benchmark the latest JetStream and transport work.
This commit is contained in:
Joseph Doherty
2026-03-14 03:13:17 -04:00
parent 56c773dc71
commit ba0d65317a
76 changed files with 3058 additions and 29987 deletions
+18 -8
View File
@@ -59,7 +59,7 @@ Benchmark run: 2026-03-13 America/Indiana/Indianapolis. Both servers ran on the
| Mode | Payload | Storage | Go msg/s | .NET msg/s | Ratio (.NET/Go) | | Mode | Payload | Storage | Go msg/s | .NET msg/s | Ratio (.NET/Go) |
|------|---------|---------|----------|------------|-----------------| |------|---------|---------|----------|------------|-----------------|
| Synchronous | 16 B | Memory | 16,982 | 14,514 | 0.85x | | Synchronous | 16 B | Memory | 16,982 | 14,514 | 0.85x |
| Async (batch) | 128 B | File | 211,355 | 58,334 | 0.28x | | Async (batch) | 128 B | File | 174,421 | 85,394 | 0.49x |
--- ---
@@ -144,7 +144,7 @@ Benchmark run: 2026-03-13 America/Indiana/Indianapolis. Both servers ran on the
| Request/reply (single) | 0.89x | Close to parity | | Request/reply (single) | 0.89x | Close to parity |
| Request/reply (10Cx2S) | 0.86x | Close to parity | | Request/reply (10Cx2S) | 0.86x | Close to parity |
| JetStream sync publish | 0.85x | Close to parity | | JetStream sync publish | 0.85x | Close to parity |
| JetStream async file publish | 0.28x | Storage-bound | | JetStream async file publish | 0.49x | Improved after double-buffer + deferred fsync |
| JetStream ordered consume | 0.44x | Significant gap | | JetStream ordered consume | 0.44x | Significant gap |
| JetStream durable fetch | 0.76x | Moderate gap | | JetStream durable fetch | 0.76x | Moderate gap |
| MQTT pub/sub | **1.32x** | .NET outperforms Go | | MQTT pub/sub | **1.32x** | .NET outperforms Go |
@@ -157,16 +157,26 @@ Benchmark run: 2026-03-13 America/Indiana/Indianapolis. Both servers ran on the
### Key Observations ### Key Observations
1. **Multi pub/sub reached parity (1.01x)** after Round 10 pre-formatted MSG headers. Fan-out improved to 0.84x. 1. **Multi pub/sub reached parity (1.01x)** after Round 10 pre-formatted MSG headers. Fan-out improved to 0.84x.
2. **TLS pub/sub shows a dramatic .NET advantage (4.70x)** — .NET's `SslStream` has significantly lower overhead in the bidirectional pub/sub path. TLS pub-only (ingest only) still favors Go at 0.39x, suggesting the advantage is in the read-and-deliver path. 2. **JetStream async file publish improved to 0.49x** (from 0.28x) after Round 11 double-buffer + deferred fsync optimizations — a 75% improvement.
3. **MQTT pub/sub remains a .NET strength at 1.32x.** Cross-protocol (NATS→MQTT) dropped to 0.71x — this benchmark shows high variance across runs. 3. **TLS pub/sub shows a dramatic .NET advantage (4.70x)** — .NET's `SslStream` has significantly lower overhead in the bidirectional pub/sub path. TLS pub-only (ingest only) still favors Go at 0.39x, suggesting the advantage is in the read-and-deliver path.
4. **JetStream ordered consumer dropped to 0.44x** compared to earlier runs (0.62x). This test completes in <100ms and shows high variance. 4. **MQTT pub/sub remains a .NET strength at 1.32x.** Cross-protocol (NATS→MQTT) dropped to 0.71x — this benchmark shows high variance across runs.
5. **Single publisher 128B dropped to 0.37x** (from 0.62x with smaller message counts). With 500K messages, this benchmark runs long enough for Go's goroutine scheduler and buffer management to reach steady state, widening the gap. The 16B variant is stable at 0.74x. 5. **JetStream ordered consumer dropped to 0.44x** compared to earlier runs (0.62x). This test completes in <100ms and shows high variance.
6. **Request-reply latency stable** at 0.86x0.89x across all runs. 6. **Single publisher 128B dropped to 0.37x** (from 0.62x with smaller message counts). With 500K messages, this benchmark runs long enough for Go's goroutine scheduler and buffer management to reach steady state, widening the gap. The 16B variant is stable at 0.74x.
7. **Request-reply latency stable** at 0.86x0.89x across all runs.
--- ---
## Optimization History ## Optimization History
### Round 11: JetStream FileStore Double-Buffer + Deferred Fsync
Two optimizations targeting the JetStream async file publish hot path (0.28x→0.49x, 75% improvement):
| # | Root Cause | Fix | Impact |
|---|-----------|-----|--------|
| 41 | **Lock contention between WriteAt and FlushPending**`MsgBlock.FlushPending()` held the write lock for the entire `RandomAccess.Write` call, blocking `WriteAt` (publish path) during disk I/O | Double-buffer: swap `_pendingBuf``_flushBuf` under write lock, then write old buffer to disk outside lock using separate `_flushLock`; publish path only blocked during buffer pointer swap, not disk I/O | Eliminates write-lock contention during disk I/O |
| 42 | **Synchronous fsync on publish path**`RotateBlock()` called `FlushToDisk()` which did `fsync` synchronously (1,557ms per profile), blocking the publish hot path for every block rotation | Deferred fsync: `RotateBlock` enqueues completed blocks into `ConcurrentQueue<MsgBlock> _needSyncBlocks`; background `FlushLoopAsync` drains the queue via `DrainSyncQueue()`, calling `Flush()` (fsync) off the publish path — matches Go's `needSync` flag + background goroutine pattern | Moves fsync entirely off the publish hot path |
### Round 10: Fan-Out Serial Path Optimization ### Round 10: Fan-Out Serial Path Optimization
Three optimizations making the serial fan-out path cheaper (fan-out 0.63x→0.84x, multi 0.65x→1.01x): Three optimizations making the serial fan-out path cheaper (fan-out 0.63x→0.84x, multi 0.65x→1.01x):
@@ -280,6 +290,6 @@ Additional fixes: SHA256 envelope bypass for unencrypted/uncompressed stores, RA
| Change | Expected Impact | Go Reference | | Change | Expected Impact | Go Reference |
|--------|----------------|-------------| |--------|----------------|-------------|
| **Single publisher ingest path (0.37x at 128B)** | The pub-only path has the largest gap. Go's readLoop uses zero-copy buffer management with direct `[]byte` slicing; .NET parses into managed objects. Reducing allocations in the parser→ProcessMessage path would help. | Go: `client.go` readLoop, direct buffer slicing | | **Single publisher ingest path (0.37x at 128B)** | The pub-only path has the largest gap. Go's readLoop uses zero-copy buffer management with direct `[]byte` slicing; .NET parses into managed objects. Reducing allocations in the parser→ProcessMessage path would help. | Go: `client.go` readLoop, direct buffer slicing |
| **JetStream async file publish (0.28x)** | Storage-bound: FileStore AppendAsync bottleneck is synchronous `RandomAccess.Write` in flush loop and S2 compression overhead | Go: `filestore.go` uses `cache.buf`/`cache.idx` with mmap and goroutine-per-flush concurrency | | **JetStream async file publish (0.49x)** | After double-buffer + deferred fsync, remaining gap is likely write coalescing and S2 compression overhead | Go: `filestore.go` uses `cache.buf`/`cache.idx` with mmap and goroutine-per-flush concurrency |
| **JetStream ordered consumer (0.44x)** | Pull consumer delivery pipeline has overhead in the fetch→deliver→ack cycle. The test completes in <100ms so numbers are noisy, but the gap is real. | Go: `consumer.go` delivery with direct buffer writes | | **JetStream ordered consumer (0.44x)** | Pull consumer delivery pipeline has overhead in the fetch→deliver→ack cycle. The test completes in <100ms so numbers are noisy, but the gap is real. | Go: `consumer.go` delivery with direct buffer writes |
| **Write-loop / socket write overhead** | Fan-out (0.84x) and pub/sub (0.66x) gaps partly come from write-loop wakeup latency and socket write syscall overhead compared to Go's `writev()` | Go: `flushOutbound` uses `net.Buffers.WriteTo``writev()` with zero-copy buffer management | | **Write-loop / socket write overhead** | Fan-out (0.84x) and pub/sub (0.66x) gaps partly come from write-loop wakeup latency and socket write syscall overhead compared to Go's `writev()` | Go: `flushOutbound` uses `net.Buffers.WriteTo``writev()` with zero-copy buffer management |
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
File diff suppressed because one or more lines are too long
Binary file not shown.
File diff suppressed because one or more lines are too long
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
+881 -11611
View File
File diff suppressed because it is too large Load Diff
+76 -18346
View File
File diff suppressed because it is too large Load Diff
@@ -33,6 +33,13 @@ public static partial class PermissionTemplates
/// Returns an empty list if any template resolves to no values (tag not found). /// Returns an empty list if any template resolves to no values (tag not found).
/// Returns a single-element list containing the original pattern if no templates are present. /// Returns a single-element list containing the original pattern if no templates are present.
/// </summary> /// </summary>
/// <param name="pattern">Template subject pattern to expand.</param>
/// <param name="name">User display name from JWT claims.</param>
/// <param name="subject">User public NKey subject from JWT claims.</param>
/// <param name="accountName">Account display name from account JWT.</param>
/// <param name="accountSubject">Account public NKey subject from account JWT.</param>
/// <param name="userTags">User tag set in <c>key:value</c> form.</param>
/// <param name="accountTags">Account tag set in <c>key:value</c> form.</param>
public static List<string> Expand( public static List<string> Expand(
string pattern, string pattern,
string name, string subject, string name, string subject,
@@ -71,6 +78,13 @@ public static partial class PermissionTemplates
/// Expands all patterns in a permission list, flattening multi-value expansions /// Expands all patterns in a permission list, flattening multi-value expansions
/// into the result. Patterns that resolve to no values are omitted entirely. /// into the result. Patterns that resolve to no values are omitted entirely.
/// </summary> /// </summary>
/// <param name="patterns">Permission subject patterns to expand.</param>
/// <param name="name">User display name from JWT claims.</param>
/// <param name="subject">User public NKey subject from JWT claims.</param>
/// <param name="accountName">Account display name from account JWT.</param>
/// <param name="accountSubject">Account public NKey subject from account JWT.</param>
/// <param name="userTags">User tag set in <c>key:value</c> form.</param>
/// <param name="accountTags">Account tag set in <c>key:value</c> form.</param>
public static List<string> ExpandAll( public static List<string> ExpandAll(
IEnumerable<string> patterns, IEnumerable<string> patterns,
string name, string subject, string name, string subject,
@@ -65,12 +65,15 @@ public sealed class RemoteLeafOptions
/// Sets reconnect/connect delay for this remote. /// Sets reconnect/connect delay for this remote.
/// Go reference: leafnode.go leafNodeCfg.setConnectDelay. /// Go reference: leafnode.go leafNodeCfg.setConnectDelay.
/// </summary> /// </summary>
/// <param name="delay">Delay before the next reconnect attempt to this remote leaf.</param>
public void SetConnectDelay(TimeSpan delay) => _connectDelay = delay; public void SetConnectDelay(TimeSpan delay) => _connectDelay = delay;
/// <summary> /// <summary>
/// Starts or replaces the JetStream migration timer callback for this remote leaf. /// Starts or replaces the JetStream migration timer callback for this remote leaf.
/// Go reference: leafnode.go leafNodeCfg.migrateTimer. /// Go reference: leafnode.go leafNodeCfg.migrateTimer.
/// </summary> /// </summary>
/// <param name="callback">Callback invoked when migration retry timing elapses.</param>
/// <param name="delay">Initial delay before invoking the migration callback.</param>
public void StartMigrateTimer(TimerCallback callback, TimeSpan delay) public void StartMigrateTimer(TimerCallback callback, TimeSpan delay)
{ {
ArgumentNullException.ThrowIfNull(callback); ArgumentNullException.ThrowIfNull(callback);
@@ -93,6 +96,7 @@ public sealed class RemoteLeafOptions
/// Saves TLS hostname from URL for future SNI usage. /// Saves TLS hostname from URL for future SNI usage.
/// Go reference: leafnode.go leafNodeCfg.saveTLSHostname. /// Go reference: leafnode.go leafNodeCfg.saveTLSHostname.
/// </summary> /// </summary>
/// <param name="url">Remote leaf URL that supplies the SNI host name.</param>
public void SaveTlsHostname(string url) public void SaveTlsHostname(string url)
{ {
if (TryParseUrl(url, out var uri)) if (TryParseUrl(url, out var uri))
@@ -103,6 +107,7 @@ public sealed class RemoteLeafOptions
/// Saves username/password from URL user info for fallback auth. /// Saves username/password from URL user info for fallback auth.
/// Go reference: leafnode.go leafNodeCfg.saveUserPassword. /// Go reference: leafnode.go leafNodeCfg.saveUserPassword.
/// </summary> /// </summary>
/// <param name="url">Remote leaf URL containing optional user info credentials.</param>
public void SaveUserPassword(string url) public void SaveUserPassword(string url)
{ {
if (!TryParseUrl(url, out var uri) || string.IsNullOrEmpty(uri.UserInfo)) if (!TryParseUrl(url, out var uri) || string.IsNullOrEmpty(uri.UserInfo))
@@ -124,18 +129,25 @@ public sealed class RemoteLeafOptions
public sealed class LeafNodeOptions public sealed class LeafNodeOptions
{ {
/// <summary>Host/IP address where the leaf listener accepts incoming leaf connections.</summary>
public string Host { get; set; } = "0.0.0.0"; public string Host { get; set; } = "0.0.0.0";
/// <summary>TCP port exposed for leaf node connections.</summary>
public int Port { get; set; } public int Port { get; set; }
// Auth for leaf listener // Auth for leaf listener
/// <summary>Optional username required for inbound leaf authentication.</summary>
public string? Username { get; set; } public string? Username { get; set; }
/// <summary>Optional password required for inbound leaf authentication.</summary>
public string? Password { get; set; } public string? Password { get; set; }
/// <summary>Maximum seconds a leaf connection has to complete authentication.</summary>
public double AuthTimeout { get; set; } public double AuthTimeout { get; set; }
// Advertise address // Advertise address
/// <summary>Optional externally reachable leaf address advertised to peers.</summary>
public string? Advertise { get; set; } public string? Advertise { get; set; }
// Per-subsystem write deadline // Per-subsystem write deadline
/// <summary>Write deadline applied to leaf network operations.</summary>
public TimeSpan WriteDeadline { get; set; } public TimeSpan WriteDeadline { get; set; }
/// <summary> /// <summary>
@@ -156,9 +168,13 @@ public sealed class LeafNodeOptions
/// </summary> /// </summary>
public string? JetStreamDomain { get; set; } public string? JetStreamDomain { get; set; }
/// <summary>Subjects that this leaf cannot export to the remote account.</summary>
public List<string> DenyExports { get; set; } = []; public List<string> DenyExports { get; set; } = [];
/// <summary>Subjects that this leaf cannot import from the remote account.</summary>
public List<string> DenyImports { get; set; } = []; public List<string> DenyImports { get; set; } = [];
/// <summary>Subjects explicitly exported from this leaf to connected remotes.</summary>
public List<string> ExportSubjects { get; set; } = []; public List<string> ExportSubjects { get; set; } = [];
/// <summary>Subjects explicitly imported from remote leaves into this server.</summary>
public List<string> ImportSubjects { get; set; } = []; public List<string> ImportSubjects { get; set; } = [];
/// <summary>List of users for leaf listener authentication (from authorization.users).</summary> /// <summary>List of users for leaf listener authentication (from authorization.users).</summary>
@@ -15,10 +15,15 @@ public sealed class GatewayConnection(Socket socket) : IAsyncDisposable
private readonly ConcurrentDictionary<string, HashSet<string>> _queueSubscriptions = new(StringComparer.Ordinal); private readonly ConcurrentDictionary<string, HashSet<string>> _queueSubscriptions = new(StringComparer.Ordinal);
private Task? _loopTask; private Task? _loopTask;
/// <summary>Remote gateway server id learned during handshake.</summary>
public string? RemoteId { get; private set; } public string? RemoteId { get; private set; }
/// <summary>Indicates whether this is an outbound (solicited) gateway connection.</summary>
public bool IsOutbound { get; internal set; } public bool IsOutbound { get; internal set; }
/// <summary>Remote endpoint string for diagnostics and monitoring.</summary>
public string RemoteEndpoint => socket.RemoteEndPoint?.ToString() ?? Guid.NewGuid().ToString("N"); public string RemoteEndpoint => socket.RemoteEndPoint?.ToString() ?? Guid.NewGuid().ToString("N");
/// <summary>Callback invoked when remote A+/A- interest updates are received.</summary>
public Func<RemoteSubscription, Task>? RemoteSubscriptionReceived { get; set; } public Func<RemoteSubscription, Task>? RemoteSubscriptionReceived { get; set; }
/// <summary>Callback invoked when remote GMSG payloads are received.</summary>
public Func<GatewayMessage, Task>? MessageReceived { get; set; } public Func<GatewayMessage, Task>? MessageReceived { get; set; }
/// <summary> /// <summary>
@@ -31,6 +36,8 @@ public sealed class GatewayConnection(Socket socket) : IAsyncDisposable
/// Adds a subject to the account-specific subscription set for this gateway connection. /// Adds a subject to the account-specific subscription set for this gateway connection.
/// Go: gateway.go — per-account subscription routing state on outbound connections. /// Go: gateway.go — per-account subscription routing state on outbound connections.
/// </summary> /// </summary>
/// <param name="account">Account name for the subscription.</param>
/// <param name="subject">Subject to track.</param>
public void AddAccountSubscription(string account, string subject) public void AddAccountSubscription(string account, string subject)
{ {
var subs = _accountSubscriptions.GetOrAdd(account, _ => new HashSet<string>(StringComparer.Ordinal)); var subs = _accountSubscriptions.GetOrAdd(account, _ => new HashSet<string>(StringComparer.Ordinal));
@@ -40,6 +47,8 @@ public sealed class GatewayConnection(Socket socket) : IAsyncDisposable
/// <summary> /// <summary>
/// Removes a subject from the account-specific subscription set for this gateway connection. /// Removes a subject from the account-specific subscription set for this gateway connection.
/// </summary> /// </summary>
/// <param name="account">Account name for the subscription.</param>
/// <param name="subject">Subject to untrack.</param>
public void RemoveAccountSubscription(string account, string subject) public void RemoveAccountSubscription(string account, string subject)
{ {
if (_accountSubscriptions.TryGetValue(account, out var subs)) if (_accountSubscriptions.TryGetValue(account, out var subs))
@@ -49,6 +58,8 @@ public sealed class GatewayConnection(Socket socket) : IAsyncDisposable
/// <summary> /// <summary>
/// Returns a snapshot of all subjects tracked for the given account on this connection. /// Returns a snapshot of all subjects tracked for the given account on this connection.
/// </summary> /// </summary>
/// <param name="account">Account name to query.</param>
/// <returns>Snapshot of tracked subjects.</returns>
public IReadOnlySet<string> GetAccountSubscriptions(string account) public IReadOnlySet<string> GetAccountSubscriptions(string account)
{ {
if (_accountSubscriptions.TryGetValue(account, out var subs)) if (_accountSubscriptions.TryGetValue(account, out var subs))
@@ -59,6 +70,8 @@ public sealed class GatewayConnection(Socket socket) : IAsyncDisposable
/// <summary> /// <summary>
/// Returns the number of subjects tracked for the given account. Returns 0 for unknown accounts. /// Returns the number of subjects tracked for the given account. Returns 0 for unknown accounts.
/// </summary> /// </summary>
/// <param name="account">Account name to query.</param>
/// <returns>Number of tracked subjects for the account.</returns>
public int AccountSubscriptionCount(string account) public int AccountSubscriptionCount(string account)
{ {
if (_accountSubscriptions.TryGetValue(account, out var subs)) if (_accountSubscriptions.TryGetValue(account, out var subs))
@@ -70,6 +83,8 @@ public sealed class GatewayConnection(Socket socket) : IAsyncDisposable
/// Registers a queue group subscription for propagation to this gateway. /// Registers a queue group subscription for propagation to this gateway.
/// Go reference: gateway.go — sendQueueSubsToGateway. /// Go reference: gateway.go — sendQueueSubsToGateway.
/// </summary> /// </summary>
/// <param name="subject">Subject for the queue subscription.</param>
/// <param name="queueGroup">Queue group name.</param>
public void AddQueueSubscription(string subject, string queueGroup) public void AddQueueSubscription(string subject, string queueGroup)
{ {
var groups = _queueSubscriptions.GetOrAdd(subject, _ => new HashSet<string>(StringComparer.Ordinal)); var groups = _queueSubscriptions.GetOrAdd(subject, _ => new HashSet<string>(StringComparer.Ordinal));
@@ -80,6 +95,8 @@ public sealed class GatewayConnection(Socket socket) : IAsyncDisposable
/// Removes a queue group subscription from this gateway connection's tracking state. /// Removes a queue group subscription from this gateway connection's tracking state.
/// Go reference: gateway.go — sendQueueSubsToGateway (removal path). /// Go reference: gateway.go — sendQueueSubsToGateway (removal path).
/// </summary> /// </summary>
/// <param name="subject">Subject for the queue subscription.</param>
/// <param name="queueGroup">Queue group name.</param>
public void RemoveQueueSubscription(string subject, string queueGroup) public void RemoveQueueSubscription(string subject, string queueGroup)
{ {
if (_queueSubscriptions.TryGetValue(subject, out var groups)) if (_queueSubscriptions.TryGetValue(subject, out var groups))
@@ -89,6 +106,8 @@ public sealed class GatewayConnection(Socket socket) : IAsyncDisposable
/// <summary> /// <summary>
/// Returns a snapshot of all queue group names registered for the given subject. /// Returns a snapshot of all queue group names registered for the given subject.
/// </summary> /// </summary>
/// <param name="subject">Subject to query.</param>
/// <returns>Snapshot of queue group names.</returns>
public IReadOnlySet<string> GetQueueGroups(string subject) public IReadOnlySet<string> GetQueueGroups(string subject)
{ {
if (_queueSubscriptions.TryGetValue(subject, out var groups)) if (_queueSubscriptions.TryGetValue(subject, out var groups))
@@ -104,6 +123,9 @@ public sealed class GatewayConnection(Socket socket) : IAsyncDisposable
/// <summary> /// <summary>
/// Returns true if the given subject/queueGroup pair is currently registered on this gateway connection. /// Returns true if the given subject/queueGroup pair is currently registered on this gateway connection.
/// </summary> /// </summary>
/// <param name="subject">Subject to query.</param>
/// <param name="queueGroup">Queue group name to query.</param>
/// <returns><see langword="true"/> when the pair is registered.</returns>
public bool HasQueueSubscription(string subject, string queueGroup) public bool HasQueueSubscription(string subject, string queueGroup)
{ {
if (!_queueSubscriptions.TryGetValue(subject, out var groups)) if (!_queueSubscriptions.TryGetValue(subject, out var groups))
@@ -111,6 +133,11 @@ public sealed class GatewayConnection(Socket socket) : IAsyncDisposable
lock (groups) return groups.Contains(queueGroup); lock (groups) return groups.Contains(queueGroup);
} }
/// <summary>
/// Performs outbound gateway handshake by sending local id and reading remote id.
/// </summary>
/// <param name="serverId">Local server id.</param>
/// <param name="ct">Cancellation token for I/O operations.</param>
public async Task PerformOutboundHandshakeAsync(string serverId, CancellationToken ct) public async Task PerformOutboundHandshakeAsync(string serverId, CancellationToken ct)
{ {
await WriteLineAsync($"GATEWAY {serverId}", ct); await WriteLineAsync($"GATEWAY {serverId}", ct);
@@ -118,6 +145,11 @@ public sealed class GatewayConnection(Socket socket) : IAsyncDisposable
RemoteId = ParseHandshake(line); RemoteId = ParseHandshake(line);
} }
/// <summary>
/// Performs inbound gateway handshake by reading remote id and sending local id.
/// </summary>
/// <param name="serverId">Local server id.</param>
/// <param name="ct">Cancellation token for I/O operations.</param>
public async Task PerformInboundHandshakeAsync(string serverId, CancellationToken ct) public async Task PerformInboundHandshakeAsync(string serverId, CancellationToken ct)
{ {
var line = await ReadLineAsync(ct); var line = await ReadLineAsync(ct);
@@ -125,6 +157,10 @@ public sealed class GatewayConnection(Socket socket) : IAsyncDisposable
await WriteLineAsync($"GATEWAY {serverId}", ct); await WriteLineAsync($"GATEWAY {serverId}", ct);
} }
/// <summary>
/// Starts the background frame read loop for this connection.
/// </summary>
/// <param name="ct">Cancellation token controlling loop lifetime.</param>
public void StartLoop(CancellationToken ct) public void StartLoop(CancellationToken ct)
{ {
if (_loopTask != null) if (_loopTask != null)
@@ -134,15 +170,42 @@ public sealed class GatewayConnection(Socket socket) : IAsyncDisposable
_loopTask = Task.Run(() => ReadLoopAsync(linked.Token), linked.Token); _loopTask = Task.Run(() => ReadLoopAsync(linked.Token), linked.Token);
} }
/// <summary>
/// Waits for the gateway read loop to exit.
/// </summary>
/// <param name="ct">Cancellation token for wait operation.</param>
/// <returns>A task that completes when loop exits.</returns>
public Task WaitUntilClosedAsync(CancellationToken ct) public Task WaitUntilClosedAsync(CancellationToken ct)
=> _loopTask?.WaitAsync(ct) ?? Task.CompletedTask; => _loopTask?.WaitAsync(ct) ?? Task.CompletedTask;
/// <summary>
/// Sends an A+ protocol line to advertise interest.
/// </summary>
/// <param name="account">Account for the interest update.</param>
/// <param name="subject">Subject being added.</param>
/// <param name="queue">Optional queue group.</param>
/// <param name="ct">Cancellation token for I/O operations.</param>
public Task SendAPlusAsync(string account, string subject, string? queue, CancellationToken ct) public Task SendAPlusAsync(string account, string subject, string? queue, CancellationToken ct)
=> WriteLineAsync(queue is { Length: > 0 } ? $"A+ {account} {subject} {queue}" : $"A+ {account} {subject}", ct); => WriteLineAsync(queue is { Length: > 0 } ? $"A+ {account} {subject} {queue}" : $"A+ {account} {subject}", ct);
/// <summary>
/// Sends an A- protocol line to remove advertised interest.
/// </summary>
/// <param name="account">Account for the interest update.</param>
/// <param name="subject">Subject being removed.</param>
/// <param name="queue">Optional queue group.</param>
/// <param name="ct">Cancellation token for I/O operations.</param>
public Task SendAMinusAsync(string account, string subject, string? queue, CancellationToken ct) public Task SendAMinusAsync(string account, string subject, string? queue, CancellationToken ct)
=> WriteLineAsync(queue is { Length: > 0 } ? $"A- {account} {subject} {queue}" : $"A- {account} {subject}", ct); => WriteLineAsync(queue is { Length: > 0 } ? $"A- {account} {subject} {queue}" : $"A- {account} {subject}", ct);
/// <summary>
/// Sends a GMSG payload to the remote gateway when interest permits forwarding.
/// </summary>
/// <param name="account">Account associated with the message.</param>
/// <param name="subject">Subject being forwarded.</param>
/// <param name="replyTo">Optional reply subject.</param>
/// <param name="payload">Payload bytes.</param>
/// <param name="ct">Cancellation token for I/O operations.</param>
public async Task SendMessageAsync(string account, string subject, string? replyTo, ReadOnlyMemory<byte> payload, CancellationToken ct) public async Task SendMessageAsync(string account, string subject, string? replyTo, ReadOnlyMemory<byte> payload, CancellationToken ct)
{ {
// Go: gateway.go:2900 (shouldForwardMsg) — check interest tracker before sending // Go: gateway.go:2900 (shouldForwardMsg) — check interest tracker before sending
@@ -166,6 +229,9 @@ public sealed class GatewayConnection(Socket socket) : IAsyncDisposable
} }
} }
/// <summary>
/// Disposes this gateway connection and stops background processing.
/// </summary>
public async ValueTask DisposeAsync() public async ValueTask DisposeAsync()
{ {
await _closedCts.CancelAsync(); await _closedCts.CancelAsync();
+49
View File
@@ -20,6 +20,8 @@ public static class ReplyMapper
/// Checks whether the subject starts with either gateway reply prefix: /// Checks whether the subject starts with either gateway reply prefix:
/// <c>_GR_.</c> (current) or <c>$GR.</c> (legacy). /// <c>_GR_.</c> (current) or <c>$GR.</c> (legacy).
/// </summary> /// </summary>
/// <param name="subject">Subject to inspect.</param>
/// <returns><see langword="true"/> when the subject is gateway-routed.</returns>
public static bool HasGatewayReplyPrefix(string? subject) public static bool HasGatewayReplyPrefix(string? subject)
=> IsGatewayRoutedSubject(subject, out _); => IsGatewayRoutedSubject(subject, out _);
@@ -28,6 +30,9 @@ public static class ReplyMapper
/// old prefix (<c>$GR.</c>) was used. /// old prefix (<c>$GR.</c>) was used.
/// Go reference: isGWRoutedSubjectAndIsOldPrefix. /// Go reference: isGWRoutedSubjectAndIsOldPrefix.
/// </summary> /// </summary>
/// <param name="subject">Subject to inspect.</param>
/// <param name="isOldPrefix">Set to <see langword="true"/> when the legacy prefix is used.</param>
/// <returns><see langword="true"/> when the subject is gateway-routed.</returns>
public static bool IsGatewayRoutedSubject(string? subject, out bool isOldPrefix) public static bool IsGatewayRoutedSubject(string? subject, out bool isOldPrefix)
{ {
isOldPrefix = false; isOldPrefix = false;
@@ -51,6 +56,8 @@ public static class ReplyMapper
/// Go reference: gateway.go uses SHA-256 truncated to base-62; we use FNV-1a for speed /// Go reference: gateway.go uses SHA-256 truncated to base-62; we use FNV-1a for speed
/// while maintaining determinism and good distribution. /// while maintaining determinism and good distribution.
/// </summary> /// </summary>
/// <param name="replyTo">Reply subject to hash.</param>
/// <returns>Non-negative deterministic hash value.</returns>
public static long ComputeReplyHash(string replyTo) public static long ComputeReplyHash(string replyTo)
{ {
// FNV-1a 64-bit // FNV-1a 64-bit
@@ -72,6 +79,8 @@ public static class ReplyMapper
/// Computes the short (6-char) gateway hash used in modern gateway reply routing. /// Computes the short (6-char) gateway hash used in modern gateway reply routing.
/// Go reference: getGWHash. /// Go reference: getGWHash.
/// </summary> /// </summary>
/// <param name="gatewayName">Gateway name to hash.</param>
/// <returns>Lowercase 6-character hash token.</returns>
public static string ComputeGatewayHash(string gatewayName) public static string ComputeGatewayHash(string gatewayName)
{ {
var digest = System.Security.Cryptography.SHA256.HashData(System.Text.Encoding.UTF8.GetBytes(gatewayName)); var digest = System.Security.Cryptography.SHA256.HashData(System.Text.Encoding.UTF8.GetBytes(gatewayName));
@@ -82,6 +91,8 @@ public static class ReplyMapper
/// Computes the short (4-char) legacy gateway hash used with old prefixes. /// Computes the short (4-char) legacy gateway hash used with old prefixes.
/// Go reference: getOldHash. /// Go reference: getOldHash.
/// </summary> /// </summary>
/// <param name="gatewayName">Gateway name to hash.</param>
/// <returns>Lowercase 4-character hash token.</returns>
public static string ComputeOldGatewayHash(string gatewayName) public static string ComputeOldGatewayHash(string gatewayName)
{ {
var digest = System.Security.Cryptography.SHA256.HashData(System.Text.Encoding.UTF8.GetBytes(gatewayName)); var digest = System.Security.Cryptography.SHA256.HashData(System.Text.Encoding.UTF8.GetBytes(gatewayName));
@@ -92,6 +103,10 @@ public static class ReplyMapper
/// Converts a reply subject to gateway form with an explicit hash segment. /// Converts a reply subject to gateway form with an explicit hash segment.
/// Format: <c>_GR_.{clusterId}.{hash}.{originalReply}</c>. /// Format: <c>_GR_.{clusterId}.{hash}.{originalReply}</c>.
/// </summary> /// </summary>
/// <param name="replyTo">Original reply subject.</param>
/// <param name="localClusterId">Local cluster identifier to embed.</param>
/// <param name="hash">Precomputed reply hash.</param>
/// <returns>Gateway-form reply subject, or original when null/empty.</returns>
public static string? ToGatewayReply(string? replyTo, string localClusterId, long hash) public static string? ToGatewayReply(string? replyTo, string localClusterId, long hash)
{ {
if (string.IsNullOrWhiteSpace(replyTo)) if (string.IsNullOrWhiteSpace(replyTo))
@@ -104,6 +119,9 @@ public static class ReplyMapper
/// Converts a reply subject to gateway form, automatically computing the hash. /// Converts a reply subject to gateway form, automatically computing the hash.
/// Format: <c>_GR_.{clusterId}.{hash}.{originalReply}</c>. /// Format: <c>_GR_.{clusterId}.{hash}.{originalReply}</c>.
/// </summary> /// </summary>
/// <param name="replyTo">Original reply subject.</param>
/// <param name="localClusterId">Local cluster identifier to embed.</param>
/// <returns>Gateway-form reply subject, or original when null/empty.</returns>
public static string? ToGatewayReply(string? replyTo, string localClusterId) public static string? ToGatewayReply(string? replyTo, string localClusterId)
{ {
if (string.IsNullOrWhiteSpace(replyTo)) if (string.IsNullOrWhiteSpace(replyTo))
@@ -119,6 +137,9 @@ public static class ReplyMapper
/// legacy format (<c>_GR_.{clusterId}.{originalReply}</c>). /// legacy format (<c>_GR_.{clusterId}.{originalReply}</c>).
/// Nested prefixes are unwrapped iteratively. /// Nested prefixes are unwrapped iteratively.
/// </summary> /// </summary>
/// <param name="gatewayReply">Gateway-form reply subject.</param>
/// <param name="restoredReply">Receives restored original reply subject on success.</param>
/// <returns><see langword="true"/> when restoration succeeds.</returns>
public static bool TryRestoreGatewayReply(string? gatewayReply, out string restoredReply) public static bool TryRestoreGatewayReply(string? gatewayReply, out string restoredReply)
{ {
restoredReply = string.Empty; restoredReply = string.Empty;
@@ -161,6 +182,9 @@ public static class ReplyMapper
/// Extracts the cluster ID from a gateway reply subject. /// Extracts the cluster ID from a gateway reply subject.
/// The cluster ID is the first segment after the <c>_GR_.</c> prefix. /// The cluster ID is the first segment after the <c>_GR_.</c> prefix.
/// </summary> /// </summary>
/// <param name="gatewayReply">Gateway-form reply subject.</param>
/// <param name="clusterId">Receives extracted cluster identifier on success.</param>
/// <returns><see langword="true"/> when extraction succeeds.</returns>
public static bool TryExtractClusterId(string? gatewayReply, out string clusterId) public static bool TryExtractClusterId(string? gatewayReply, out string clusterId)
{ {
clusterId = string.Empty; clusterId = string.Empty;
@@ -181,6 +205,9 @@ public static class ReplyMapper
/// Extracts the hash from a gateway reply subject (new format only). /// Extracts the hash from a gateway reply subject (new format only).
/// Returns false if the reply uses the legacy format without a hash. /// Returns false if the reply uses the legacy format without a hash.
/// </summary> /// </summary>
/// <param name="gatewayReply">Gateway-form reply subject.</param>
/// <param name="hash">Receives extracted hash on success.</param>
/// <returns><see langword="true"/> when extraction succeeds.</returns>
public static bool TryExtractHash(string? gatewayReply, out long hash) public static bool TryExtractHash(string? gatewayReply, out long hash)
{ {
hash = 0; hash = 0;
@@ -236,6 +263,11 @@ public sealed class ReplyMapCache
private long _hits; private long _hits;
private long _misses; private long _misses;
/// <summary>
/// Creates an LRU reply mapping cache with TTL expiration.
/// </summary>
/// <param name="capacity">Maximum number of entries to retain.</param>
/// <param name="ttlMs">Time-to-live for entries in milliseconds.</param>
public ReplyMapCache(int capacity = 4096, int ttlMs = 60_000) public ReplyMapCache(int capacity = 4096, int ttlMs = 60_000)
{ {
_capacity = capacity; _capacity = capacity;
@@ -243,10 +275,19 @@ public sealed class ReplyMapCache
_map = new Dictionary<string, LinkedListNode<CacheEntry>>(capacity, StringComparer.Ordinal); _map = new Dictionary<string, LinkedListNode<CacheEntry>>(capacity, StringComparer.Ordinal);
} }
/// <summary>Total cache hits since creation.</summary>
public long Hits => Interlocked.Read(ref _hits); public long Hits => Interlocked.Read(ref _hits);
/// <summary>Total cache misses since creation.</summary>
public long Misses => Interlocked.Read(ref _misses); public long Misses => Interlocked.Read(ref _misses);
/// <summary>Current number of entries in the cache.</summary>
public int Count { get { lock (_lock) return _map.Count; } } public int Count { get { lock (_lock) return _map.Count; } }
/// <summary>
/// Attempts to get a cached mapping value.
/// </summary>
/// <param name="key">Cache lookup key.</param>
/// <param name="value">Resolved cached value when found and not expired.</param>
/// <returns><see langword="true"/> when an unexpired value exists.</returns>
public bool TryGet(string key, out string? value) public bool TryGet(string key, out string? value)
{ {
lock (_lock) lock (_lock)
@@ -276,6 +317,11 @@ public sealed class ReplyMapCache
return false; return false;
} }
/// <summary>
/// Inserts or updates a cached mapping value.
/// </summary>
/// <param name="key">Cache key.</param>
/// <param name="value">Cache value.</param>
public void Set(string key, string value) public void Set(string key, string value)
{ {
lock (_lock) lock (_lock)
@@ -302,6 +348,9 @@ public sealed class ReplyMapCache
} }
} }
/// <summary>
/// Clears all cached mappings.
/// </summary>
public void Clear() public void Clear()
{ {
lock (_lock) lock (_lock)
@@ -18,10 +18,17 @@ public static class GslErrors
/// </summary> /// </summary>
internal sealed class Level<T> where T : IEquatable<T> internal sealed class Level<T> where T : IEquatable<T>
{ {
/// <summary>Literal-token child nodes.</summary>
public Dictionary<string, Node<T>> Nodes { get; } = new(); public Dictionary<string, Node<T>> Nodes { get; } = new();
/// <summary>Single-token wildcard child node (<c>*</c>).</summary>
public Node<T>? Pwc { get; set; } // partial wildcard '*' public Node<T>? Pwc { get; set; } // partial wildcard '*'
/// <summary>Terminal wildcard child node (<c>&gt;</c>).</summary>
public Node<T>? Fwc { get; set; } // full wildcard '>' public Node<T>? Fwc { get; set; } // full wildcard '>'
/// <summary>
/// Returns the number of child node references at this level, including wildcard pointers.
/// </summary>
/// <returns>Child count for this level.</returns>
public int NumNodes() public int NumNodes()
{ {
var num = Nodes.Count; var num = Nodes.Count;
@@ -34,6 +41,8 @@ internal sealed class Level<T> where T : IEquatable<T>
/// Prune an empty node from the tree. /// Prune an empty node from the tree.
/// Go reference: server/gsl/gsl.go pruneNode /// Go reference: server/gsl/gsl.go pruneNode
/// </summary> /// </summary>
/// <param name="n">Node to prune.</param>
/// <param name="token">Token key used for literal node lookup.</param>
public void PruneNode(Node<T> n, string token) public void PruneNode(Node<T> n, string token)
{ {
if (ReferenceEquals(n, Fwc)) if (ReferenceEquals(n, Fwc))
@@ -51,7 +60,9 @@ internal sealed class Level<T> where T : IEquatable<T>
/// </summary> /// </summary>
internal sealed class Node<T> where T : IEquatable<T> internal sealed class Node<T> where T : IEquatable<T>
{ {
/// <summary>Next trie level for descendant tokens.</summary>
public Level<T>? Next { get; set; } public Level<T>? Next { get; set; }
/// <summary>Subscriptions stored on this node keyed by value.</summary>
public Dictionary<T, string> Subs { get; } = new(); // value -> subject public Dictionary<T, string> Subs { get; } = new(); // value -> subject
/// <summary> /// <summary>
@@ -107,6 +118,8 @@ public class GenericSubjectList<T> where T : IEquatable<T>
/// Insert adds a subscription into the sublist. /// Insert adds a subscription into the sublist.
/// Go reference: server/gsl/gsl.go Insert /// Go reference: server/gsl/gsl.go Insert
/// </summary> /// </summary>
/// <param name="subject">Subject to insert.</param>
/// <param name="value">Subscription payload/value.</param>
public void Insert(string subject, T value) public void Insert(string subject, T value)
{ {
_lock.EnterWriteLock(); _lock.EnterWriteLock();
@@ -185,6 +198,8 @@ public class GenericSubjectList<T> where T : IEquatable<T>
/// Remove will remove a subscription. /// Remove will remove a subscription.
/// Go reference: server/gsl/gsl.go Remove /// Go reference: server/gsl/gsl.go Remove
/// </summary> /// </summary>
/// <param name="subject">Subject to remove.</param>
/// <param name="value">Subscription payload/value to remove.</param>
public void Remove(string subject, T value) public void Remove(string subject, T value)
{ {
_lock.EnterWriteLock(); _lock.EnterWriteLock();
@@ -202,6 +217,8 @@ public class GenericSubjectList<T> where T : IEquatable<T>
/// Match will match all entries to the literal subject and invoke the callback for each. /// Match will match all entries to the literal subject and invoke the callback for each.
/// Go reference: server/gsl/gsl.go Match /// Go reference: server/gsl/gsl.go Match
/// </summary> /// </summary>
/// <param name="subject">Literal subject to match.</param>
/// <param name="callback">Callback invoked for each matched value.</param>
public void Match(string subject, Action<T> callback) public void Match(string subject, Action<T> callback)
{ {
MatchInternal(subject, callback, doLock: true); MatchInternal(subject, callback, doLock: true);
@@ -211,6 +228,8 @@ public class GenericSubjectList<T> where T : IEquatable<T>
/// MatchBytes will match all entries to the literal subject (as bytes) and invoke the callback for each. /// MatchBytes will match all entries to the literal subject (as bytes) and invoke the callback for each.
/// Go reference: server/gsl/gsl.go MatchBytes /// Go reference: server/gsl/gsl.go MatchBytes
/// </summary> /// </summary>
/// <param name="subject">Literal subject bytes to match.</param>
/// <param name="callback">Callback invoked for each matched value.</param>
public void MatchBytes(ReadOnlySpan<byte> subject, Action<T> callback) public void MatchBytes(ReadOnlySpan<byte> subject, Action<T> callback)
{ {
// Convert bytes to string then delegate // Convert bytes to string then delegate
@@ -222,6 +241,8 @@ public class GenericSubjectList<T> where T : IEquatable<T>
/// HasInterest will return whether or not there is any interest in the subject. /// HasInterest will return whether or not there is any interest in the subject.
/// Go reference: server/gsl/gsl.go HasInterest /// Go reference: server/gsl/gsl.go HasInterest
/// </summary> /// </summary>
/// <param name="subject">Literal subject to test.</param>
/// <returns><see langword="true"/> when any subscription matches.</returns>
public bool HasInterest(string subject) public bool HasInterest(string subject)
{ {
return HasInterestInternal(subject, doLock: true, np: null); return HasInterestInternal(subject, doLock: true, np: null);
@@ -231,6 +252,8 @@ public class GenericSubjectList<T> where T : IEquatable<T>
/// NumInterest will return the number of subs interested in the subject. /// NumInterest will return the number of subs interested in the subject.
/// Go reference: server/gsl/gsl.go NumInterest /// Go reference: server/gsl/gsl.go NumInterest
/// </summary> /// </summary>
/// <param name="subject">Literal subject to test.</param>
/// <returns>Number of matched subscriptions.</returns>
public int NumInterest(string subject) public int NumInterest(string subject)
{ {
var np = new int[1]; // use array to pass by reference var np = new int[1]; // use array to pass by reference
@@ -242,6 +265,8 @@ public class GenericSubjectList<T> where T : IEquatable<T>
/// HasInterestStartingIn is a helper for subject tree intersection. /// HasInterestStartingIn is a helper for subject tree intersection.
/// Go reference: server/gsl/gsl.go HasInterestStartingIn /// Go reference: server/gsl/gsl.go HasInterestStartingIn
/// </summary> /// </summary>
/// <param name="subject">Subject prefix to test.</param>
/// <returns><see langword="true"/> when interest exists beneath the prefix.</returns>
public bool HasInterestStartingIn(string subject) public bool HasInterestStartingIn(string subject)
{ {
_lock.EnterReadLock(); _lock.EnterReadLock();
@@ -602,8 +627,16 @@ public class GenericSubjectList<T> where T : IEquatable<T>
{ {
private readonly string _subject; private readonly string _subject;
/// <summary>
/// Creates a split enumerable over a subject string.
/// </summary>
/// <param name="subject">Subject to tokenize.</param>
public SplitEnumerable(string subject) => _subject = subject; public SplitEnumerable(string subject) => _subject = subject;
/// <summary>
/// Creates an enumerator for token iteration.
/// </summary>
/// <returns>Tokenizer enumerator.</returns>
public SplitEnumerator GetEnumerator() => new(_subject); public SplitEnumerator GetEnumerator() => new(_subject);
} }
@@ -613,6 +646,10 @@ public class GenericSubjectList<T> where T : IEquatable<T>
private int _start; private int _start;
private bool _done; private bool _done;
/// <summary>
/// Creates a tokenizer enumerator over the provided subject.
/// </summary>
/// <param name="subject">Subject to tokenize.</param>
public SplitEnumerator(string subject) public SplitEnumerator(string subject)
{ {
_subject = subject; _subject = subject;
@@ -621,8 +658,13 @@ public class GenericSubjectList<T> where T : IEquatable<T>
Current = default!; Current = default!;
} }
/// <summary>Current token from the subject split iteration.</summary>
public string Current { get; private set; } public string Current { get; private set; }
/// <summary>
/// Advances to the next token in the subject string.
/// </summary>
/// <returns><see langword="true"/> when another token is available.</returns>
public bool MoveNext() public bool MoveNext()
{ {
if (_done) return false; if (_done) return false;
@@ -31,6 +31,9 @@ public class HashWheel
private long _lowest; private long _lowest;
private ulong _count; private ulong _count;
/// <summary>
/// Initializes an empty time hash wheel used for expiration scheduling.
/// </summary>
public HashWheel() public HashWheel()
{ {
_wheel = new Slot?[WheelSize]; _wheel = new Slot?[WheelSize];
@@ -56,6 +59,8 @@ public class HashWheel
/// Schedules a new timer task. If the sequence already exists in the target slot, /// Schedules a new timer task. If the sequence already exists in the target slot,
/// its expiration is updated without incrementing the count. /// its expiration is updated without incrementing the count.
/// </summary> /// </summary>
/// <param name="seq">Sequence identifier for the tracked item.</param>
/// <param name="expires">Absolute expiration timestamp in nanoseconds.</param>
// Go: Add server/thw/thw.go:79 // Go: Add server/thw/thw.go:79
public void Add(ulong seq, long expires) public void Add(ulong seq, long expires)
{ {
@@ -88,6 +93,8 @@ public class HashWheel
/// Removes a timer task. Returns true if the task was found and removed, /// Removes a timer task. Returns true if the task was found and removed,
/// false if the task was not found. /// false if the task was not found.
/// </summary> /// </summary>
/// <param name="seq">Sequence identifier for the tracked item.</param>
/// <param name="expires">Previously scheduled expiration timestamp in nanoseconds.</param>
// Go: Remove server/thw/thw.go:103 // Go: Remove server/thw/thw.go:103
public bool Remove(ulong seq, long expires) public bool Remove(ulong seq, long expires)
{ {
@@ -119,6 +126,9 @@ public class HashWheel
/// Updates the expiration time of an existing timer task by removing it from /// Updates the expiration time of an existing timer task by removing it from
/// the old slot and adding it to the new one. /// the old slot and adding it to the new one.
/// </summary> /// </summary>
/// <param name="seq">Sequence identifier for the tracked item.</param>
/// <param name="oldExpires">Previous expiration timestamp in nanoseconds.</param>
/// <param name="newExpires">New expiration timestamp in nanoseconds.</param>
// Go: Update server/thw/thw.go:123 // Go: Update server/thw/thw.go:123
public void Update(ulong seq, long oldExpires, long newExpires) public void Update(ulong seq, long oldExpires, long newExpires)
{ {
@@ -131,6 +141,7 @@ public class HashWheel
/// expired entry's sequence and expiration time. If the callback returns true, /// expired entry's sequence and expiration time. If the callback returns true,
/// the entry is removed; if false, it remains for future expiration checks. /// the entry is removed; if false, it remains for future expiration checks.
/// </summary> /// </summary>
/// <param name="callback">Callback invoked for each expired entry; return true to remove it.</param>
// Go: ExpireTasks server/thw/thw.go:133 // Go: ExpireTasks server/thw/thw.go:133
public void ExpireTasks(Func<ulong, long, bool> callback) public void ExpireTasks(Func<ulong, long, bool> callback)
{ {
@@ -144,6 +155,8 @@ public class HashWheel
/// Internal expiration method that accepts an explicit timestamp. /// Internal expiration method that accepts an explicit timestamp.
/// Used by tests that need deterministic time control. /// Used by tests that need deterministic time control.
/// </summary> /// </summary>
/// <param name="ts">Current timestamp in nanoseconds used as expiration cutoff.</param>
/// <param name="callback">Callback invoked for each expired entry; return true to remove it.</param>
// Go: expireTasks server/thw/thw.go:138 // Go: expireTasks server/thw/thw.go:138
internal void ExpireTasksInternal(long ts, Func<ulong, long, bool> callback) internal void ExpireTasksInternal(long ts, Func<ulong, long, bool> callback)
{ {
@@ -215,6 +228,7 @@ public class HashWheel
/// Returns the earliest expiration time if it is before the given time. /// Returns the earliest expiration time if it is before the given time.
/// Returns <see cref="long.MaxValue"/> if no expirations exist before the specified time. /// Returns <see cref="long.MaxValue"/> if no expirations exist before the specified time.
/// </summary> /// </summary>
/// <param name="before">Upper time bound in nanoseconds.</param>
// Go: GetNextExpiration server/thw/thw.go:182 // Go: GetNextExpiration server/thw/thw.go:182
public long GetNextExpiration(long before) public long GetNextExpiration(long before)
{ {
@@ -231,6 +245,7 @@ public class HashWheel
/// The high sequence number is included and will be returned on decode. /// The high sequence number is included and will be returned on decode.
/// Format: [1 byte magic version][8 bytes entry count][8 bytes highSeq][varint expires, uvarint seq pairs...] /// Format: [1 byte magic version][8 bytes entry count][8 bytes highSeq][varint expires, uvarint seq pairs...]
/// </summary> /// </summary>
/// <param name="highSeq">High watermark sequence stored alongside wheel state.</param>
// Go: Encode server/thw/thw.go:197 // Go: Encode server/thw/thw.go:197
public byte[] Encode(ulong highSeq) public byte[] Encode(ulong highSeq)
{ {
@@ -278,6 +293,7 @@ public class HashWheel
/// Decodes a binary-encoded snapshot and replaces the contents of this wheel. /// Decodes a binary-encoded snapshot and replaces the contents of this wheel.
/// Returns the high sequence number from the snapshot and the number of bytes consumed. /// Returns the high sequence number from the snapshot and the number of bytes consumed.
/// </summary> /// </summary>
/// <param name="buf">Encoded wheel snapshot buffer.</param>
// Go: Decode server/thw/thw.go:216 // Go: Decode server/thw/thw.go:216
public (ulong HighSeq, int BytesRead) Decode(ReadOnlySpan<byte> buf) public (ulong HighSeq, int BytesRead) Decode(ReadOnlySpan<byte> buf)
{ {
@@ -412,9 +428,11 @@ public class HashWheel
internal sealed class Slot internal sealed class Slot
{ {
// Go: slot.entries — map of sequence to expires. // Go: slot.entries — map of sequence to expires.
/// <summary>Entries assigned to this wheel slot keyed by sequence identifier.</summary>
public Dictionary<ulong, long> Entries { get; } = new(); public Dictionary<ulong, long> Entries { get; } = new();
// Go: slot.lowest — lowest expiration time in this slot. // Go: slot.lowest — lowest expiration time in this slot.
/// <summary>Earliest expiration timestamp present in this slot.</summary>
public long Lowest { get; set; } = long.MaxValue; public long Lowest { get; set; } = long.MaxValue;
} }
} }
+36
View File
@@ -11,11 +11,17 @@ namespace NATS.Server;
/// </summary> /// </summary>
public sealed class InternalClient : INatsClient public sealed class InternalClient : INatsClient
{ {
/// <summary>Unique internal client identifier used in server-side bookkeeping.</summary>
public ulong Id { get; } public ulong Id { get; }
/// <summary>Internal client kind (SYSTEM, ACCOUNT, JETSTREAM, etc.).</summary>
public ClientKind Kind { get; } public ClientKind Kind { get; }
/// <summary>Indicates this client is server-internal and not backed by a socket connection.</summary>
public bool IsInternal => Kind.IsInternal(); public bool IsInternal => Kind.IsInternal();
/// <summary>Account context associated with this internal client.</summary>
public Account? Account { get; } public Account? Account { get; }
/// <summary>Client options are not applicable for socketless internal clients.</summary>
public ClientOptions? ClientOpts => null; public ClientOptions? ClientOpts => null;
/// <summary>Permission overrides are not used for internal clients.</summary>
public ClientPermissions? Permissions => null; public ClientPermissions? Permissions => null;
/// <summary> /// <summary>
@@ -26,6 +32,12 @@ public sealed class InternalClient : INatsClient
private readonly Dictionary<string, Subscription> _subs = new(StringComparer.Ordinal); private readonly Dictionary<string, Subscription> _subs = new(StringComparer.Ordinal);
/// <summary>
/// Creates a lightweight internal client used for in-process message delivery.
/// </summary>
/// <param name="id">Server-assigned internal client identifier.</param>
/// <param name="kind">Client kind that must represent an internal role.</param>
/// <param name="account">Account context used for subscription accounting.</param>
public InternalClient(ulong id, ClientKind kind, Account account) public InternalClient(ulong id, ClientKind kind, Account account)
{ {
if (!kind.IsInternal()) if (!kind.IsInternal())
@@ -36,12 +48,28 @@ public sealed class InternalClient : INatsClient
Account = account; Account = account;
} }
/// <summary>
/// Delivers a message to this internal client and flushes immediately through the callback path.
/// </summary>
/// <param name="subject">Message subject.</param>
/// <param name="sid">Subscription identifier receiving the message.</param>
/// <param name="replyTo">Optional reply subject for request-reply flows.</param>
/// <param name="headers">Serialized NATS headers payload.</param>
/// <param name="payload">Message payload bytes.</param>
public void SendMessage(string subject, string sid, string? replyTo, public void SendMessage(string subject, string sid, string? replyTo,
ReadOnlyMemory<byte> headers, ReadOnlyMemory<byte> payload) ReadOnlyMemory<byte> headers, ReadOnlyMemory<byte> payload)
{ {
MessageCallback?.Invoke(subject, sid, replyTo, headers, payload); MessageCallback?.Invoke(subject, sid, replyTo, headers, payload);
} }
/// <summary>
/// Delivers a message without explicit flush semantics; equivalent to <see cref="SendMessage"/>.
/// </summary>
/// <param name="subject">Message subject.</param>
/// <param name="sid">Subscription identifier receiving the message.</param>
/// <param name="replyTo">Optional reply subject for request-reply flows.</param>
/// <param name="headers">Serialized NATS headers payload.</param>
/// <param name="payload">Message payload bytes.</param>
public void SendMessageNoFlush(string subject, string sid, string? replyTo, public void SendMessageNoFlush(string subject, string sid, string? replyTo,
ReadOnlyMemory<byte> headers, ReadOnlyMemory<byte> payload) ReadOnlyMemory<byte> headers, ReadOnlyMemory<byte> payload)
{ {
@@ -49,20 +77,28 @@ public sealed class InternalClient : INatsClient
MessageCallback?.Invoke(subject, sid, replyTo, headers, payload); MessageCallback?.Invoke(subject, sid, replyTo, headers, payload);
} }
/// <summary>Signals flush completion for interface compatibility; no-op for internal clients.</summary>
public void SignalFlush() { } // no-op for internal clients public void SignalFlush() { } // no-op for internal clients
/// <summary>Queues outbound bytes for interface compatibility; always succeeds for internal clients.</summary>
/// <param name="data">Serialized protocol bytes.</param>
public bool QueueOutbound(ReadOnlyMemory<byte> data) => true; // no-op for internal clients public bool QueueOutbound(ReadOnlyMemory<byte> data) => true; // no-op for internal clients
/// <summary>Removes a subscription and updates account subscription counters.</summary>
/// <param name="sid">Subscription identifier to remove.</param>
public void RemoveSubscription(string sid) public void RemoveSubscription(string sid)
{ {
if (_subs.Remove(sid)) if (_subs.Remove(sid))
Account?.DecrementSubscriptions(); Account?.DecrementSubscriptions();
} }
/// <summary>Adds or replaces a subscription tracked by this internal client.</summary>
/// <param name="sub">Subscription to register.</param>
public void AddSubscription(Subscription sub) public void AddSubscription(Subscription sub)
{ {
_subs[sub.Sid] = sub; _subs[sub.Sid] = sub;
} }
/// <summary>Current subscription map keyed by subscription identifier.</summary>
public IReadOnlyDictionary<string, Subscription> Subscriptions => _subs; public IReadOnlyDictionary<string, Subscription> Subscriptions => _subs;
} }
@@ -17,6 +17,12 @@ public static class ConsumerApiHandlers
private const string UnpinPrefix = JetStreamApiSubjects.ConsumerUnpin; private const string UnpinPrefix = JetStreamApiSubjects.ConsumerUnpin;
private const string NextPrefix = JetStreamApiSubjects.ConsumerNext; private const string NextPrefix = JetStreamApiSubjects.ConsumerNext;
/// <summary>
/// Handles consumer create/update requests by parsing subject and config payload.
/// </summary>
/// <param name="subject">Consumer create API subject.</param>
/// <param name="payload">JSON payload containing consumer configuration.</param>
/// <param name="consumerManager">Consumer manager responsible for create/update operations.</param>
public static JetStreamApiResponse HandleCreate(string subject, ReadOnlySpan<byte> payload, ConsumerManager consumerManager) public static JetStreamApiResponse HandleCreate(string subject, ReadOnlySpan<byte> payload, ConsumerManager consumerManager)
{ {
var parsed = ParseSubject(subject, CreatePrefix); var parsed = ParseSubject(subject, CreatePrefix);
@@ -31,6 +37,11 @@ public static class ConsumerApiHandlers
return consumerManager.CreateOrUpdate(stream, config); return consumerManager.CreateOrUpdate(stream, config);
} }
/// <summary>
/// Handles consumer info requests for a specific stream/durable pair.
/// </summary>
/// <param name="subject">Consumer info API subject.</param>
/// <param name="consumerManager">Consumer manager responsible for lookup operations.</param>
public static JetStreamApiResponse HandleInfo(string subject, ConsumerManager consumerManager) public static JetStreamApiResponse HandleInfo(string subject, ConsumerManager consumerManager)
{ {
var parsed = ParseSubject(subject, InfoPrefix); var parsed = ParseSubject(subject, InfoPrefix);
@@ -41,6 +52,11 @@ public static class ConsumerApiHandlers
return consumerManager.GetInfo(stream, durableName); return consumerManager.GetInfo(stream, durableName);
} }
/// <summary>
/// Handles consumer delete requests for a specific stream/durable pair.
/// </summary>
/// <param name="subject">Consumer delete API subject.</param>
/// <param name="consumerManager">Consumer manager responsible for deletion.</param>
public static JetStreamApiResponse HandleDelete(string subject, ConsumerManager consumerManager) public static JetStreamApiResponse HandleDelete(string subject, ConsumerManager consumerManager)
{ {
var parsed = ParseSubject(subject, DeletePrefix); var parsed = ParseSubject(subject, DeletePrefix);
@@ -53,6 +69,12 @@ public static class ConsumerApiHandlers
: JetStreamApiResponse.NotFound(subject); : JetStreamApiResponse.NotFound(subject);
} }
/// <summary>
/// Handles paginated consumer name listing requests for a stream.
/// </summary>
/// <param name="subject">Consumer names API subject.</param>
/// <param name="payload">JSON payload containing pagination offset.</param>
/// <param name="consumerManager">Consumer manager used to list names.</param>
public static JetStreamApiResponse HandleNames(string subject, ReadOnlySpan<byte> payload, ConsumerManager consumerManager) public static JetStreamApiResponse HandleNames(string subject, ReadOnlySpan<byte> payload, ConsumerManager consumerManager)
{ {
var stream = ParseStreamSubject(subject, NamesPrefix); var stream = ParseStreamSubject(subject, NamesPrefix);
@@ -70,6 +92,12 @@ public static class ConsumerApiHandlers
}; };
} }
/// <summary>
/// Handles paginated consumer info listing requests for a stream.
/// </summary>
/// <param name="subject">Consumer list API subject.</param>
/// <param name="payload">JSON payload containing pagination offset.</param>
/// <param name="consumerManager">Consumer manager used to list consumer infos.</param>
public static JetStreamApiResponse HandleList(string subject, ReadOnlySpan<byte> payload, ConsumerManager consumerManager) public static JetStreamApiResponse HandleList(string subject, ReadOnlySpan<byte> payload, ConsumerManager consumerManager)
{ {
var stream = ParseStreamSubject(subject, ListPrefix); var stream = ParseStreamSubject(subject, ListPrefix);
@@ -105,6 +133,12 @@ public static class ConsumerApiHandlers
return 0; return 0;
} }
/// <summary>
/// Handles pause/resume requests for a specific consumer.
/// </summary>
/// <param name="subject">Consumer pause API subject.</param>
/// <param name="payload">JSON payload containing pause settings.</param>
/// <param name="consumerManager">Consumer manager responsible for pause state.</param>
public static JetStreamApiResponse HandlePause(string subject, ReadOnlySpan<byte> payload, ConsumerManager consumerManager) public static JetStreamApiResponse HandlePause(string subject, ReadOnlySpan<byte> payload, ConsumerManager consumerManager)
{ {
var parsed = ParseSubject(subject, PausePrefix); var parsed = ParseSubject(subject, PausePrefix);
@@ -131,6 +165,11 @@ public static class ConsumerApiHandlers
consumerManager.GetPauseUntil(stream, durableName)); consumerManager.GetPauseUntil(stream, durableName));
} }
/// <summary>
/// Handles consumer reset requests.
/// </summary>
/// <param name="subject">Consumer reset API subject.</param>
/// <param name="consumerManager">Consumer manager responsible for reset.</param>
public static JetStreamApiResponse HandleReset(string subject, ConsumerManager consumerManager) public static JetStreamApiResponse HandleReset(string subject, ConsumerManager consumerManager)
{ {
var parsed = ParseSubject(subject, ResetPrefix); var parsed = ParseSubject(subject, ResetPrefix);
@@ -143,6 +182,11 @@ public static class ConsumerApiHandlers
: JetStreamApiResponse.NotFound(subject); : JetStreamApiResponse.NotFound(subject);
} }
/// <summary>
/// Handles consumer unpin requests for pinned priority consumers.
/// </summary>
/// <param name="subject">Consumer unpin API subject.</param>
/// <param name="consumerManager">Consumer manager responsible for pin state.</param>
public static JetStreamApiResponse HandleUnpin(string subject, ConsumerManager consumerManager) public static JetStreamApiResponse HandleUnpin(string subject, ConsumerManager consumerManager)
{ {
var parsed = ParseSubject(subject, UnpinPrefix); var parsed = ParseSubject(subject, UnpinPrefix);
@@ -155,6 +199,13 @@ public static class ConsumerApiHandlers
: JetStreamApiResponse.NotFound(subject); : JetStreamApiResponse.NotFound(subject);
} }
/// <summary>
/// Handles pull-consumer next batch requests and returns fetched messages.
/// </summary>
/// <param name="subject">Consumer next API subject.</param>
/// <param name="payload">JSON payload containing pull request options.</param>
/// <param name="consumerManager">Consumer manager responsible for fetch operations.</param>
/// <param name="streamManager">Stream manager used to resolve stream stores.</param>
public static JetStreamApiResponse HandleNext(string subject, ReadOnlySpan<byte> payload, ConsumerManager consumerManager, StreamManager streamManager) public static JetStreamApiResponse HandleNext(string subject, ReadOnlySpan<byte> payload, ConsumerManager consumerManager, StreamManager streamManager)
{ {
var parsed = ParseSubject(subject, NextPrefix); var parsed = ParseSubject(subject, NextPrefix);
@@ -191,6 +242,10 @@ public static class ConsumerApiHandlers
/// <see cref="JetStreamMetaGroup.ProposeCreateConsumerValidatedAsync"/>. /// <see cref="JetStreamMetaGroup.ProposeCreateConsumerValidatedAsync"/>.
/// Go reference: jetstream_cluster.go:8100 jsClusteredConsumerRequest. /// Go reference: jetstream_cluster.go:8100 jsClusteredConsumerRequest.
/// </summary> /// </summary>
/// <param name="subject">Consumer create API subject.</param>
/// <param name="payload">Serialized consumer create request payload.</param>
/// <param name="metaGroup">Meta-group coordinator that validates leadership and proposes RAFT changes.</param>
/// <param name="ct">Cancellation token for RAFT proposal and validation operations.</param>
public static async Task<JetStreamApiResponse> HandleClusteredCreateAsync( public static async Task<JetStreamApiResponse> HandleClusteredCreateAsync(
string subject, string subject,
byte[] payload, byte[] payload,
@@ -230,6 +285,9 @@ public static class ConsumerApiHandlers
/// <see cref="JetStreamMetaGroup.ProposeDeleteConsumerValidatedAsync"/>. /// <see cref="JetStreamMetaGroup.ProposeDeleteConsumerValidatedAsync"/>.
/// Go reference: jetstream_cluster.go jsClusteredConsumerDeleteRequest. /// Go reference: jetstream_cluster.go jsClusteredConsumerDeleteRequest.
/// </summary> /// </summary>
/// <param name="subject">Consumer delete API subject.</param>
/// <param name="metaGroup">Meta-group coordinator that validates leadership and proposes RAFT changes.</param>
/// <param name="ct">Cancellation token for RAFT proposal and validation operations.</param>
public static async Task<JetStreamApiResponse> HandleClusteredDeleteAsync( public static async Task<JetStreamApiResponse> HandleClusteredDeleteAsync(
string subject, string subject,
JetStreamMetaGroup metaGroup, JetStreamMetaGroup metaGroup,
@@ -14,6 +14,10 @@ public interface ILeaderForwarder
/// Returns the leader's response, or null when forwarding is not available /// Returns the leader's response, or null when forwarding is not available
/// (e.g. no route to leader) so the caller can fall back to a NotLeader error. /// (e.g. no route to leader) so the caller can fall back to a NotLeader error.
/// </summary> /// </summary>
/// <param name="subject">JetStream API subject identifying the requested operation.</param>
/// <param name="payload">Serialized request payload forwarded to the leader.</param>
/// <param name="leaderName">Current meta-group leader name used as forwarding target.</param>
/// <param name="ct">Cancellation token for caller-driven request cancellation.</param>
Task<JetStreamApiResponse?> ForwardAsync( Task<JetStreamApiResponse?> ForwardAsync(
string subject, string subject,
ReadOnlyMemory<byte> payload, ReadOnlyMemory<byte> payload,
@@ -36,6 +40,10 @@ public sealed class DefaultLeaderForwarder
/// </summary> /// </summary>
public TimeSpan Timeout { get; } public TimeSpan Timeout { get; }
/// <summary>
/// Creates a default leader forwarder with an optional forwarding timeout.
/// </summary>
/// <param name="timeout">Leader forwarding timeout; defaults to five seconds when omitted.</param>
public DefaultLeaderForwarder(TimeSpan? timeout = null) public DefaultLeaderForwarder(TimeSpan? timeout = null)
{ {
Timeout = timeout ?? TimeSpan.FromSeconds(5); Timeout = timeout ?? TimeSpan.FromSeconds(5);
@@ -73,11 +81,21 @@ public sealed class JetStreamApiRouter
private readonly ILeaderForwarder? _forwarder; private readonly ILeaderForwarder? _forwarder;
private long _forwardedCount; private long _forwardedCount;
/// <summary>
/// Creates a router with default in-memory managers and no clustering metadata.
/// </summary>
public JetStreamApiRouter() public JetStreamApiRouter()
: this(new StreamManager(), new ConsumerManager(), null) : this(new StreamManager(), new ConsumerManager(), null)
{ {
} }
/// <summary>
/// Creates a router with explicit managers and optional cluster leader-forwarding dependencies.
/// </summary>
/// <param name="streamManager">Stream manager handling stream API operations.</param>
/// <param name="consumerManager">Consumer manager handling consumer API operations.</param>
/// <param name="metaGroup">Optional meta-group used for leader checks and leader identity.</param>
/// <param name="forwarder">Optional forwarder used to proxy leader-only requests.</param>
public JetStreamApiRouter( public JetStreamApiRouter(
StreamManager streamManager, StreamManager streamManager,
ConsumerManager consumerManager, ConsumerManager consumerManager,
@@ -104,6 +122,7 @@ public sealed class JetStreamApiRouter
/// Read-only operations (Info, Names, List, MessageGet, Snapshot, DirectGet, Next) do not. /// Read-only operations (Info, Names, List, MessageGet, Snapshot, DirectGet, Next) do not.
/// Go reference: jetstream_api.go:200-300. /// Go reference: jetstream_api.go:200-300.
/// </summary> /// </summary>
/// <param name="subject">JetStream API subject to classify as leader-only or local-safe.</param>
public static bool IsLeaderRequired(string subject) public static bool IsLeaderRequired(string subject)
{ {
// Stream mutating operations // Stream mutating operations
@@ -165,6 +184,9 @@ public sealed class JetStreamApiRouter
/// Async callers should use <see cref="RouteAsync"/> which also attempts forwarding. /// Async callers should use <see cref="RouteAsync"/> which also attempts forwarding.
/// Go reference: jetstream_api.go — jsClusteredStreamXxxRequest helpers. /// Go reference: jetstream_api.go — jsClusteredStreamXxxRequest helpers.
/// </summary> /// </summary>
/// <param name="subject">JetStream API subject being routed.</param>
/// <param name="payload">Serialized request payload.</param>
/// <param name="leaderName">Known current leader name returned in the not-leader response.</param>
public static JetStreamApiResponse ForwardToLeader(string subject, ReadOnlySpan<byte> payload, string leaderName) public static JetStreamApiResponse ForwardToLeader(string subject, ReadOnlySpan<byte> payload, string leaderName)
{ {
_ = subject; _ = subject;
@@ -181,6 +203,9 @@ public sealed class JetStreamApiRouter
/// Read-only operations are always handled locally regardless of leadership. /// Read-only operations are always handled locally regardless of leadership.
/// Go reference: jetstream_api.go:200-300 — leader-forwarding path. /// Go reference: jetstream_api.go:200-300 — leader-forwarding path.
/// </summary> /// </summary>
/// <param name="subject">JetStream API subject to dispatch.</param>
/// <param name="payload">Request payload bytes for the API operation.</param>
/// <param name="ct">Cancellation token for asynchronous routing and forwarding.</param>
public async Task<JetStreamApiResponse> RouteAsync( public async Task<JetStreamApiResponse> RouteAsync(
string subject, string subject,
ReadOnlyMemory<byte> payload, ReadOnlyMemory<byte> payload,
@@ -219,6 +244,11 @@ public sealed class JetStreamApiRouter
return Route(subject, payload.Span); return Route(subject, payload.Span);
} }
/// <summary>
/// Routes a JetStream API request synchronously to local handlers or not-leader fallback.
/// </summary>
/// <param name="subject">JetStream API subject to dispatch.</param>
/// <param name="payload">Request payload bytes for handler execution.</param>
public JetStreamApiResponse Route(string subject, ReadOnlySpan<byte> payload) public JetStreamApiResponse Route(string subject, ReadOnlySpan<byte> payload)
{ {
// Go reference: jetstream_api.go:200-300 — leader check + forwarding. // Go reference: jetstream_api.go:200-300 — leader check + forwarding.
@@ -35,7 +35,9 @@ public sealed class AckProcessor
private int _maxDeliver; private int _maxDeliver;
private readonly List<ulong> _exceededSequences = new(); private readonly List<ulong> _exceededSequences = new();
/// <summary>Highest contiguous acknowledged consumer sequence.</summary>
public ulong AckFloor { get; private set; } public ulong AckFloor { get; private set; }
/// <summary>Number of sequences terminated with +TERM or delivery-limit exhaustion.</summary>
public int TerminatedCount { get; private set; } public int TerminatedCount { get; private set; }
/// <summary> /// <summary>
@@ -60,12 +62,21 @@ public sealed class AckProcessor
/// <summary>Policy applied when a sequence exceeds its max delivery count.</summary> /// <summary>Policy applied when a sequence exceeds its max delivery count.</summary>
public DeliveryExceededPolicy ExceededPolicy { get; set; } = DeliveryExceededPolicy.Drop; public DeliveryExceededPolicy ExceededPolicy { get; set; } = DeliveryExceededPolicy.Drop;
/// <summary>
/// Creates an ack processor with optional redelivery backoff schedule.
/// </summary>
/// <param name="backoffMs">Optional per-delivery backoff delays in milliseconds.</param>
public AckProcessor(int[]? backoffMs = null) public AckProcessor(int[]? backoffMs = null)
{ {
_backoffMs = backoffMs; _backoffMs = backoffMs;
} }
// Go: consumer.go — ConsumerConfig maxAckPending + RedeliveryTracker integration // Go: consumer.go — ConsumerConfig maxAckPending + RedeliveryTracker integration
/// <summary>
/// Creates an ack processor that integrates with a redelivery tracker.
/// </summary>
/// <param name="tracker">Redelivery tracker used for delivery metadata.</param>
/// <param name="maxAckPending">Maximum pending acknowledgements; 0 means unlimited.</param>
public AckProcessor(RedeliveryTracker tracker, int maxAckPending = 0) public AckProcessor(RedeliveryTracker tracker, int maxAckPending = 0)
{ {
_tracker = tracker; _tracker = tracker;
@@ -74,6 +85,11 @@ public sealed class AckProcessor
_backoffMs = null; _backoffMs = null;
} }
/// <summary>
/// Registers a delivered sequence with an acknowledgement deadline.
/// </summary>
/// <param name="sequence">Consumer sequence delivered to a client.</param>
/// <param name="ackWaitMs">Ack wait timeout in milliseconds.</param>
public void Register(ulong sequence, int ackWaitMs) public void Register(ulong sequence, int ackWaitMs)
{ {
if (sequence <= AckFloor) if (sequence <= AckFloor)
@@ -92,6 +108,11 @@ public sealed class AckProcessor
} }
// Go: consumer.go — register with deliver subject; ackWait comes from the tracker // Go: consumer.go — register with deliver subject; ackWait comes from the tracker
/// <summary>
/// Registers a delivered sequence with tracker-provided ack wait and deliver subject.
/// </summary>
/// <param name="sequence">Consumer sequence delivered to a client.</param>
/// <param name="deliverSubject">Deliver subject associated with the sequence.</param>
public void Register(ulong sequence, string deliverSubject) public void Register(ulong sequence, string deliverSubject)
{ {
if (_tracker is null) if (_tracker is null)
@@ -105,6 +126,10 @@ public sealed class AckProcessor
} }
// Go: consumer.go — processAck without payload: plain +ACK, also notifies tracker // Go: consumer.go — processAck without payload: plain +ACK, also notifies tracker
/// <summary>
/// Processes a plain acknowledgement for a sequence.
/// </summary>
/// <param name="seq">Acknowledged sequence.</param>
public void ProcessAck(ulong seq) public void ProcessAck(ulong seq)
{ {
AckSequence(seq); AckSequence(seq);
@@ -112,6 +137,11 @@ public sealed class AckProcessor
} }
// Go: consumer.go — returns ack deadline for a pending sequence; MinValue if not tracked // Go: consumer.go — returns ack deadline for a pending sequence; MinValue if not tracked
/// <summary>
/// Gets the current acknowledgement deadline for a pending sequence.
/// </summary>
/// <param name="seq">Sequence to query.</param>
/// <returns>Deadline in UTC offset form, or <see cref="DateTimeOffset.MinValue"/> when unknown.</returns>
public DateTimeOffset GetDeadline(ulong seq) public DateTimeOffset GetDeadline(ulong seq)
{ {
if (_pending.TryGetValue(seq, out var state)) if (_pending.TryGetValue(seq, out var state))
@@ -121,9 +151,18 @@ public sealed class AckProcessor
} }
// Go: consumer.go — maxAckPending=0 means unlimited; otherwise cap pending registrations // Go: consumer.go — maxAckPending=0 means unlimited; otherwise cap pending registrations
/// <summary>
/// Indicates whether another pending sequence can be registered.
/// </summary>
/// <returns><see langword="true"/> when pending registrations are below max limits.</returns>
public bool CanRegister() => _maxAckPending <= 0 || _pending.Count < _maxAckPending; public bool CanRegister() => _maxAckPending <= 0 || _pending.Count < _maxAckPending;
// Go: consumer.go:2550 — parse ack type prefix from raw payload bytes // Go: consumer.go:2550 — parse ack type prefix from raw payload bytes
/// <summary>
/// Parses an ack payload and returns its ack type.
/// </summary>
/// <param name="data">Raw ack payload bytes.</param>
/// <returns>Detected ack type.</returns>
public static AckType ParseAckType(ReadOnlySpan<byte> data) public static AckType ParseAckType(ReadOnlySpan<byte> data)
{ {
if (data.StartsWith("+ACK"u8)) if (data.StartsWith("+ACK"u8))
@@ -137,6 +176,12 @@ public sealed class AckProcessor
return AckType.Unknown; return AckType.Unknown;
} }
/// <summary>
/// Finds one pending sequence whose ack deadline has expired.
/// </summary>
/// <param name="sequence">Receives expired sequence when found.</param>
/// <param name="deliveries">Receives current delivery count for the sequence.</param>
/// <returns><see langword="true"/> when an expired sequence is found.</returns>
public bool TryGetExpired(out ulong sequence, out int deliveries) public bool TryGetExpired(out ulong sequence, out int deliveries)
{ {
foreach (var (seq, state) in _pending) foreach (var (seq, state) in _pending)
@@ -157,6 +202,11 @@ public sealed class AckProcessor
// Go: consumer.go:2550 (processAck) // Go: consumer.go:2550 (processAck)
// Dispatches to the appropriate ack handler based on ack type prefix. // Dispatches to the appropriate ack handler based on ack type prefix.
// Empty or "+ACK" → ack single; "-NAK" → schedule redelivery; "+TERM" → terminate; "+WPI" → progress reset. // Empty or "+ACK" → ack single; "-NAK" → schedule redelivery; "+TERM" → terminate; "+WPI" → progress reset.
/// <summary>
/// Processes an ack payload and dispatches to ack/nak/term/progress handling.
/// </summary>
/// <param name="seq">Sequence associated with the ack payload.</param>
/// <param name="payload">Raw ack payload bytes.</param>
public void ProcessAck(ulong seq, ReadOnlySpan<byte> payload) public void ProcessAck(ulong seq, ReadOnlySpan<byte> payload)
{ {
if (payload.IsEmpty || payload.SequenceEqual("+ACK"u8)) if (payload.IsEmpty || payload.SequenceEqual("+ACK"u8))
@@ -197,6 +247,10 @@ public sealed class AckProcessor
} }
// Go: consumer.go — processAck for "+ACK": removes from pending and advances AckFloor when contiguous // Go: consumer.go — processAck for "+ACK": removes from pending and advances AckFloor when contiguous
/// <summary>
/// Acknowledges a sequence and advances ack floor when possible.
/// </summary>
/// <param name="seq">Sequence to acknowledge.</param>
public void AckSequence(ulong seq) public void AckSequence(ulong seq)
{ {
_pending.Remove(seq); _pending.Remove(seq);
@@ -221,6 +275,11 @@ public sealed class AckProcessor
} }
// Go: consumer.go — processNak: schedules redelivery with optional explicit delay or backoff array // Go: consumer.go — processNak: schedules redelivery with optional explicit delay or backoff array
/// <summary>
/// Processes a NAK and schedules redelivery.
/// </summary>
/// <param name="seq">Sequence to redeliver.</param>
/// <param name="delayMs">Optional explicit redelivery delay in milliseconds.</param>
public void ProcessNak(ulong seq, int delayMs = 0) public void ProcessNak(ulong seq, int delayMs = 0)
{ {
if (_terminated.Contains(seq)) if (_terminated.Contains(seq))
@@ -249,6 +308,10 @@ public sealed class AckProcessor
} }
// Go: consumer.go — processTerm: removes from pending permanently; sequence is never redelivered // Go: consumer.go — processTerm: removes from pending permanently; sequence is never redelivered
/// <summary>
/// Processes a TERM ack, permanently terminating redelivery for the sequence.
/// </summary>
/// <param name="seq">Sequence to terminate.</param>
public void ProcessTerm(ulong seq) public void ProcessTerm(ulong seq)
{ {
if (_pending.Remove(seq)) if (_pending.Remove(seq))
@@ -259,6 +322,10 @@ public sealed class AckProcessor
} }
// Go: consumer.go — processAckProgress (+WPI): resets ack deadline to original ackWait without bumping delivery count // Go: consumer.go — processAckProgress (+WPI): resets ack deadline to original ackWait without bumping delivery count
/// <summary>
/// Processes an in-progress ack (+WPI) by extending the sequence deadline.
/// </summary>
/// <param name="seq">Sequence to extend.</param>
public void ProcessProgress(ulong seq) public void ProcessProgress(ulong seq)
{ {
if (!_pending.TryGetValue(seq, out var state)) if (!_pending.TryGetValue(seq, out var state))
@@ -268,6 +335,11 @@ public sealed class AckProcessor
_pending[seq] = state; _pending[seq] = state;
} }
/// <summary>
/// Schedules the next redelivery deadline for a sequence.
/// </summary>
/// <param name="sequence">Sequence to reschedule.</param>
/// <param name="delayMs">Redelivery delay in milliseconds.</param>
public void ScheduleRedelivery(ulong sequence, int delayMs) public void ScheduleRedelivery(ulong sequence, int delayMs)
{ {
if (!_pending.TryGetValue(sequence, out var state)) if (!_pending.TryGetValue(sequence, out var state))
@@ -289,6 +361,10 @@ public sealed class AckProcessor
_pending[sequence] = state; _pending[sequence] = state;
} }
/// <summary>
/// Drops a pending sequence without advancing ack floor.
/// </summary>
/// <param name="sequence">Sequence to remove from pending state.</param>
public void Drop(ulong sequence) public void Drop(ulong sequence)
{ {
_pending.Remove(sequence); _pending.Remove(sequence);
@@ -312,6 +388,7 @@ public sealed class AckProcessor
/// Resets the ack floor to the specified value. /// Resets the ack floor to the specified value.
/// Used during consumer reset. /// Used during consumer reset.
/// </summary> /// </summary>
/// <param name="floor">New ack floor value.</param>
public void SetAckFloor(ulong floor) public void SetAckFloor(ulong floor)
{ {
AckFloor = floor; AckFloor = floor;
@@ -320,9 +397,15 @@ public sealed class AckProcessor
_pending.Remove(key); _pending.Remove(key);
} }
/// <summary>Indicates whether there are pending unacked sequences.</summary>
public bool HasPending => _pending.Count > 0; public bool HasPending => _pending.Count > 0;
/// <summary>Current number of pending unacked sequences.</summary>
public int PendingCount => _pending.Count; public int PendingCount => _pending.Count;
/// <summary>
/// Acknowledges all pending sequences up to and including the provided sequence.
/// </summary>
/// <param name="sequence">Highest sequence to acknowledge.</param>
public void AckAll(ulong sequence) public void AckAll(ulong sequence)
{ {
foreach (var key in _pending.Keys.Where(k => k <= sequence).ToArray()) foreach (var key in _pending.Keys.Where(k => k <= sequence).ToArray())
@@ -359,7 +442,9 @@ public sealed class AckProcessor
private sealed class PendingState private sealed class PendingState
{ {
/// <summary>Current acknowledgement deadline in UTC.</summary>
public DateTime DeadlineUtc { get; set; } public DateTime DeadlineUtc { get; set; }
/// <summary>Current delivery attempt count.</summary>
public int Deliveries { get; set; } public int Deliveries { get; set; }
} }
} }
@@ -19,6 +19,9 @@ public sealed class PriorityGroupManager
/// Register a consumer in a named priority group. /// Register a consumer in a named priority group.
/// Lower <paramref name="priority"/> values indicate higher priority. /// Lower <paramref name="priority"/> values indicate higher priority.
/// </summary> /// </summary>
/// <param name="groupName">Priority group name used to coordinate active consumer selection.</param>
/// <param name="consumerId">Consumer identifier to register in the group.</param>
/// <param name="priority">Priority rank where lower numbers are favored.</param>
public void Register(string groupName, string consumerId, int priority) public void Register(string groupName, string consumerId, int priority)
{ {
var group = _groups.GetOrAdd(groupName, _ => new PriorityGroup()); var group = _groups.GetOrAdd(groupName, _ => new PriorityGroup());
@@ -41,6 +44,8 @@ public sealed class PriorityGroupManager
/// <summary> /// <summary>
/// Remove a consumer from a named priority group. /// Remove a consumer from a named priority group.
/// </summary> /// </summary>
/// <param name="groupName">Priority group name.</param>
/// <param name="consumerId">Consumer identifier to remove.</param>
public void Unregister(string groupName, string consumerId) public void Unregister(string groupName, string consumerId)
{ {
if (!_groups.TryGetValue(groupName, out var group)) if (!_groups.TryGetValue(groupName, out var group))
@@ -61,6 +66,7 @@ public sealed class PriorityGroupManager
/// in the named group, or <c>null</c> if the group is empty or does not exist. /// in the named group, or <c>null</c> if the group is empty or does not exist.
/// When multiple consumers share the same lowest priority, the first registered wins. /// When multiple consumers share the same lowest priority, the first registered wins.
/// </summary> /// </summary>
/// <param name="groupName">Priority group name.</param>
public string? GetActiveConsumer(string groupName) public string? GetActiveConsumer(string groupName)
{ {
if (!_groups.TryGetValue(groupName, out var group)) if (!_groups.TryGetValue(groupName, out var group))
@@ -86,6 +92,8 @@ public sealed class PriorityGroupManager
/// Returns <c>true</c> if the given consumer is the current active consumer /// Returns <c>true</c> if the given consumer is the current active consumer
/// (lowest priority number) in the named group. /// (lowest priority number) in the named group.
/// </summary> /// </summary>
/// <param name="groupName">Priority group name.</param>
/// <param name="consumerId">Consumer identifier to validate.</param>
public bool IsActive(string groupName, string consumerId) public bool IsActive(string groupName, string consumerId)
{ {
var active = GetActiveConsumer(groupName); var active = GetActiveConsumer(groupName);
@@ -96,6 +104,8 @@ public sealed class PriorityGroupManager
/// Assign a new pin ID to the named group, replacing any existing pin. /// Assign a new pin ID to the named group, replacing any existing pin.
/// Go reference: consumer.go (assignNewPinId). /// Go reference: consumer.go (assignNewPinId).
/// </summary> /// </summary>
/// <param name="groupName">Priority group name.</param>
/// <param name="consumerId">Consumer being pinned as active (reserved for API parity).</param>
/// <returns>The newly generated 22-character pin ID.</returns> /// <returns>The newly generated 22-character pin ID.</returns>
public string AssignPinId(string groupName, string consumerId) public string AssignPinId(string groupName, string consumerId)
{ {
@@ -115,6 +125,8 @@ public sealed class PriorityGroupManager
/// Returns <c>true</c> if the group exists and its current pin ID equals <paramref name="pinId"/>. /// Returns <c>true</c> if the group exists and its current pin ID equals <paramref name="pinId"/>.
/// Go reference: consumer.go (setPinnedTimer). /// Go reference: consumer.go (setPinnedTimer).
/// </summary> /// </summary>
/// <param name="groupName">Priority group name.</param>
/// <param name="pinId">Pin identifier to validate.</param>
public bool ValidatePinId(string groupName, string pinId) public bool ValidatePinId(string groupName, string pinId)
{ {
if (!_groups.TryGetValue(groupName, out var group)) if (!_groups.TryGetValue(groupName, out var group))
@@ -131,6 +143,7 @@ public sealed class PriorityGroupManager
/// Clear the current pin ID for the named group. No-op if the group does not exist. /// Clear the current pin ID for the named group. No-op if the group does not exist.
/// Go reference: consumer.go (setPinnedTimer). /// Go reference: consumer.go (setPinnedTimer).
/// </summary> /// </summary>
/// <param name="groupName">Priority group name.</param>
public void UnassignPinId(string groupName) public void UnassignPinId(string groupName)
{ {
if (!_groups.TryGetValue(groupName, out var group)) if (!_groups.TryGetValue(groupName, out var group))
@@ -144,8 +157,11 @@ public sealed class PriorityGroupManager
private sealed class PriorityGroup private sealed class PriorityGroup
{ {
/// <summary>Synchronization gate for mutable group membership and pin state.</summary>
public object Lock { get; } = new(); public object Lock { get; } = new();
/// <summary>Registered consumers participating in this priority group.</summary>
public List<PriorityMember> Members { get; } = []; public List<PriorityMember> Members { get; } = [];
/// <summary>Current sticky pin identifier used for temporary assignment stability.</summary>
public string? CurrentPinId { get; set; } public string? CurrentPinId { get; set; }
} }
@@ -25,6 +25,7 @@ public enum ConsumerSignal
public sealed class PushConsumerEngine public sealed class PushConsumerEngine
{ {
// Go: consumer.go — DeliverSubject routes push-mode messages (cfg.DeliverSubject) // Go: consumer.go — DeliverSubject routes push-mode messages (cfg.DeliverSubject)
/// <summary>Deliver subject used for push-based consumer message delivery.</summary>
public string DeliverSubject { get; private set; } = string.Empty; public string DeliverSubject { get; private set; } = string.Empty;
private CancellationTokenSource? _cts; private CancellationTokenSource? _cts;
@@ -83,6 +84,11 @@ public sealed class PushConsumerEngine
FlowControlPendingCount--; FlowControlPendingCount--;
} }
/// <summary>
/// Enqueues data and optional control frames for a message selected for push delivery.
/// </summary>
/// <param name="consumer">Consumer receiving the queued frames.</param>
/// <param name="message">Stored stream message selected for delivery.</param>
public void Enqueue(ConsumerHandle consumer, StoredMessage message) public void Enqueue(ConsumerHandle consumer, StoredMessage message)
{ {
if (message.Sequence <= consumer.AckProcessor.AckFloor) if (message.Sequence <= consumer.AckProcessor.AckFloor)
@@ -131,6 +137,12 @@ public sealed class PushConsumerEngine
// StartDeliveryLoop wires the background pump that drains PushFrames and calls // StartDeliveryLoop wires the background pump that drains PushFrames and calls
// sendMessage for each frame. The delegate matches the wire-level send signature used // sendMessage for each frame. The delegate matches the wire-level send signature used
// by NatsClient.SendMessage, mapped to an async ValueTask for testability. // by NatsClient.SendMessage, mapped to an async ValueTask for testability.
/// <summary>
/// Starts the background delivery loop that drains push frames to the subscriber callback.
/// </summary>
/// <param name="consumer">Consumer whose queued frames are delivered.</param>
/// <param name="sendMessage">Callback that publishes a prepared frame to the target subscriber.</param>
/// <param name="ct">Cancellation token used to stop delivery.</param>
public void StartDeliveryLoop( public void StartDeliveryLoop(
ConsumerHandle consumer, ConsumerHandle consumer,
Func<string, string, ReadOnlyMemory<byte>, ReadOnlyMemory<byte>, CancellationToken, ValueTask> sendMessage, Func<string, string, ReadOnlyMemory<byte>, ReadOnlyMemory<byte>, CancellationToken, ValueTask> sendMessage,
@@ -154,6 +166,9 @@ public sealed class PushConsumerEngine
} }
} }
/// <summary>
/// Stops the delivery loop and heartbeat timer for this push engine.
/// </summary>
public void StopDeliveryLoop() public void StopDeliveryLoop()
{ {
StopIdleHeartbeatTimer(); StopIdleHeartbeatTimer();
@@ -166,6 +181,10 @@ public sealed class PushConsumerEngine
/// Starts the gather loop that polls the store for new messages. /// Starts the gather loop that polls the store for new messages.
/// Go reference: consumer.go:1400 loopAndGatherMsgs. /// Go reference: consumer.go:1400 loopAndGatherMsgs.
/// </summary> /// </summary>
/// <param name="consumer">Consumer whose delivery cursor is advanced by the gather loop.</param>
/// <param name="store">Stream store used to load pending and new messages.</param>
/// <param name="sendMessage">Callback used to emit selected messages downstream.</param>
/// <param name="ct">Cancellation token that stops gather processing.</param>
public void StartGatherLoop( public void StartGatherLoop(
ConsumerHandle consumer, ConsumerHandle consumer,
IStreamStore store, IStreamStore store,
@@ -194,6 +213,7 @@ public sealed class PushConsumerEngine
/// Signals the gather loop to wake up and re-poll the store. /// Signals the gather loop to wake up and re-poll the store.
/// Go reference: consumer.go:1620 — channel send wakes the loop. /// Go reference: consumer.go:1620 — channel send wakes the loop.
/// </summary> /// </summary>
/// <param name="signal">Reason code for waking the gather loop.</param>
public void Signal(ConsumerSignal signal) public void Signal(ConsumerSignal signal)
{ {
_signalChannel?.Writer.TryWrite(signal); _signalChannel?.Writer.TryWrite(signal);
@@ -203,6 +223,8 @@ public sealed class PushConsumerEngine
/// Public test accessor for the filter predicate. Production code uses /// Public test accessor for the filter predicate. Production code uses
/// the private ShouldDeliver; this entry point avoids reflection in unit tests. /// the private ShouldDeliver; this entry point avoids reflection in unit tests.
/// </summary> /// </summary>
/// <param name="config">Consumer filter configuration.</param>
/// <param name="subject">Candidate stream subject.</param>
public static bool ShouldDeliverPublic(ConsumerConfig config, string subject) public static bool ShouldDeliverPublic(ConsumerConfig config, string subject)
=> ShouldDeliver(config, subject); => ShouldDeliver(config, subject);
@@ -462,10 +484,15 @@ public sealed class PushConsumerEngine
public sealed class PushFrame public sealed class PushFrame
{ {
/// <summary>Indicates this frame carries stream data.</summary>
public bool IsData { get; init; } public bool IsData { get; init; }
/// <summary>Indicates this frame is a flow-control marker.</summary>
public bool IsFlowControl { get; init; } public bool IsFlowControl { get; init; }
/// <summary>Indicates this frame is an idle-heartbeat marker.</summary>
public bool IsHeartbeat { get; init; } public bool IsHeartbeat { get; init; }
/// <summary>Stored message payload for data frames; null for control frames.</summary>
public StoredMessage? Message { get; init; } public StoredMessage? Message { get; init; }
/// <summary>Earliest UTC time the frame may be emitted, used for rate limiting.</summary>
public DateTime AvailableAtUtc { get; init; } = DateTime.UtcNow; public DateTime AvailableAtUtc { get; init; } = DateTime.UtcNow;
/// <summary> /// <summary>
@@ -23,6 +23,10 @@ public sealed class RedeliveryTracker
private readonly long _ackWaitMs; private readonly long _ackWaitMs;
// Go: consumer.go:100 — BackOff []time.Duration in ConsumerConfig; empty falls back to ackWait // Go: consumer.go:100 — BackOff []time.Duration in ConsumerConfig; empty falls back to ackWait
/// <summary>
/// Initializes redelivery tracking with integer backoff delays in milliseconds.
/// </summary>
/// <param name="backoffMs">Backoff schedule where index maps to delivery attempt count.</param>
public RedeliveryTracker(int[] backoffMs) public RedeliveryTracker(int[] backoffMs)
{ {
_backoffMs = backoffMs; _backoffMs = backoffMs;
@@ -32,6 +36,12 @@ public sealed class RedeliveryTracker
} }
// Go: consumer.go — ConsumerConfig maxDeliver + ackWait + backoff, new overload storing config fields // Go: consumer.go — ConsumerConfig maxDeliver + ackWait + backoff, new overload storing config fields
/// <summary>
/// Initializes redelivery tracking using max deliveries, default ack wait, and optional long backoff schedule.
/// </summary>
/// <param name="maxDeliveries">Maximum deliveries before a message is considered exhausted.</param>
/// <param name="ackWaitMs">Default ack-wait timeout in milliseconds when no backoff entry exists.</param>
/// <param name="backoffMs">Optional per-delivery backoff schedule in milliseconds.</param>
public RedeliveryTracker(int maxDeliveries, long ackWaitMs, long[]? backoffMs = null) public RedeliveryTracker(int maxDeliveries, long ackWaitMs, long[]? backoffMs = null)
{ {
_backoffMs = []; _backoffMs = [];
@@ -43,6 +53,12 @@ public sealed class RedeliveryTracker
// Go: consumer.go:5540 — trackPending records delivery count and schedules deadline // Go: consumer.go:5540 — trackPending records delivery count and schedules deadline
// using the backoff array indexed by (deliveryCount-1), clamped at last entry. // using the backoff array indexed by (deliveryCount-1), clamped at last entry.
// Returns the UTC time at which the sequence next becomes eligible for redelivery. // Returns the UTC time at which the sequence next becomes eligible for redelivery.
/// <summary>
/// Schedules a sequence for redelivery based on delivery count and ack wait/backoff policy.
/// </summary>
/// <param name="seq">Stream sequence to track.</param>
/// <param name="deliveryCount">Current delivery attempt count.</param>
/// <param name="ackWaitMs">Ack wait fallback in milliseconds.</param>
public DateTime Schedule(ulong seq, int deliveryCount, int ackWaitMs = 0) public DateTime Schedule(ulong seq, int deliveryCount, int ackWaitMs = 0)
{ {
var delayMs = ResolveDelay(deliveryCount, ackWaitMs); var delayMs = ResolveDelay(deliveryCount, ackWaitMs);
@@ -58,6 +74,11 @@ public sealed class RedeliveryTracker
} }
// Go: consumer.go — schedule with an explicit deadline into the priority queue // Go: consumer.go — schedule with an explicit deadline into the priority queue
/// <summary>
/// Schedules a sequence for redelivery at an explicit deadline.
/// </summary>
/// <param name="seq">Stream sequence to track.</param>
/// <param name="deadline">UTC deadline when the sequence becomes due.</param>
public void Schedule(ulong seq, DateTimeOffset deadline) public void Schedule(ulong seq, DateTimeOffset deadline)
{ {
_deliveryCounts.TryAdd(seq, 0); _deliveryCounts.TryAdd(seq, 0);
@@ -65,6 +86,9 @@ public sealed class RedeliveryTracker
} }
// Go: consumer.go — rdq entries are dispatched once their deadline has passed // Go: consumer.go — rdq entries are dispatched once their deadline has passed
/// <summary>
/// Returns all tracked sequences that are currently due for redelivery.
/// </summary>
public IReadOnlyList<ulong> GetDue() public IReadOnlyList<ulong> GetDue()
{ {
var now = DateTime.UtcNow; var now = DateTime.UtcNow;
@@ -84,6 +108,10 @@ public sealed class RedeliveryTracker
// Go: consumer.go — drain the rdq priority queue of all entries whose deadline <= now, // Go: consumer.go — drain the rdq priority queue of all entries whose deadline <= now,
// returning them in deadline order (earliest first). // returning them in deadline order (earliest first).
/// <summary>
/// Returns sequences due at or before the supplied timestamp in deadline order.
/// </summary>
/// <param name="now">Current UTC time used as the due cutoff.</param>
public IEnumerable<ulong> GetDue(DateTimeOffset now) public IEnumerable<ulong> GetDue(DateTimeOffset now)
{ {
List<(ulong seq, DateTimeOffset deadline)>? dequeued = null; List<(ulong seq, DateTimeOffset deadline)>? dequeued = null;
@@ -123,6 +151,10 @@ public sealed class RedeliveryTracker
} }
// Go: consumer.go — acking a sequence removes it from the pending redelivery set // Go: consumer.go — acking a sequence removes it from the pending redelivery set
/// <summary>
/// Removes a tracked sequence after acknowledgement.
/// </summary>
/// <param name="seq">Stream sequence to stop tracking.</param>
public void Acknowledge(ulong seq) public void Acknowledge(ulong seq)
{ {
_entries.Remove(seq); _entries.Remove(seq);
@@ -131,6 +163,11 @@ public sealed class RedeliveryTracker
} }
// Go: consumer.go — maxdeliver check: drop sequence once delivery count exceeds max // Go: consumer.go — maxdeliver check: drop sequence once delivery count exceeds max
/// <summary>
/// Indicates whether a sequence has reached the supplied max-deliver threshold.
/// </summary>
/// <param name="seq">Stream sequence to inspect.</param>
/// <param name="maxDeliver">Maximum allowed delivery count.</param>
public bool IsMaxDeliveries(ulong seq, int maxDeliver) public bool IsMaxDeliveries(ulong seq, int maxDeliver)
{ {
if (maxDeliver <= 0) if (maxDeliver <= 0)
@@ -143,6 +180,10 @@ public sealed class RedeliveryTracker
} }
// Go: consumer.go — maxdeliver check using the stored _maxDeliveries from new constructor // Go: consumer.go — maxdeliver check using the stored _maxDeliveries from new constructor
/// <summary>
/// Indicates whether a sequence has reached the configured max-deliver threshold.
/// </summary>
/// <param name="seq">Stream sequence to inspect.</param>
public bool IsMaxDeliveries(ulong seq) public bool IsMaxDeliveries(ulong seq)
{ {
if (_maxDeliveries <= 0) if (_maxDeliveries <= 0)
@@ -153,6 +194,10 @@ public sealed class RedeliveryTracker
} }
// Go: consumer.go — rdc map increment: track how many times a sequence has been delivered // Go: consumer.go — rdc map increment: track how many times a sequence has been delivered
/// <summary>
/// Increments delivery attempt count for a tracked sequence.
/// </summary>
/// <param name="seq">Stream sequence to increment.</param>
public void IncrementDeliveryCount(ulong seq) public void IncrementDeliveryCount(ulong seq)
{ {
_deliveryCounts[seq] = _deliveryCounts.TryGetValue(seq, out var count) ? count + 1 : 1; _deliveryCounts[seq] = _deliveryCounts.TryGetValue(seq, out var count) ? count + 1 : 1;
@@ -160,6 +205,10 @@ public sealed class RedeliveryTracker
// Go: consumer.go — backoff delay lookup: index by deliveryCount, clamp to last entry, // Go: consumer.go — backoff delay lookup: index by deliveryCount, clamp to last entry,
// fall back to ackWait when no backoff array is configured. // fall back to ackWait when no backoff array is configured.
/// <summary>
/// Returns the redelivery backoff delay for a delivery attempt.
/// </summary>
/// <param name="deliveryCount">Delivery attempt count (1-based).</param>
public long GetBackoffDelay(int deliveryCount) public long GetBackoffDelay(int deliveryCount)
{ {
if (_backoffMsLong is { Length: > 0 }) if (_backoffMsLong is { Length: > 0 })
@@ -172,8 +221,13 @@ public sealed class RedeliveryTracker
return _ackWaitMs; return _ackWaitMs;
} }
/// <summary>
/// Indicates whether a sequence is currently tracked for redelivery.
/// </summary>
/// <param name="seq">Stream sequence to check.</param>
public bool IsTracking(ulong seq) => _entries.ContainsKey(seq); public bool IsTracking(ulong seq) => _entries.ContainsKey(seq);
/// <summary>Total number of sequences currently tracked for redelivery.</summary>
public int TrackedCount => _entries.Count; public int TrackedCount => _entries.Count;
// Go: consumer.go — backoff index = min(deliveries-1, len(backoff)-1); // Go: consumer.go — backoff index = min(deliveries-1, len(backoff)-1);
@@ -191,7 +245,9 @@ public sealed class RedeliveryTracker
private sealed class RedeliveryEntry private sealed class RedeliveryEntry
{ {
/// <summary>UTC deadline when this sequence should be considered due.</summary>
public DateTime DeadlineUtc { get; set; } public DateTime DeadlineUtc { get; set; }
/// <summary>Number of delivery attempts made for this sequence.</summary>
public int DeliveryCount { get; set; } public int DeliveryCount { get; set; }
} }
} }
@@ -6,9 +6,13 @@ namespace NATS.Server.JetStream;
/// </summary> /// </summary>
public sealed class JetStreamApiStats public sealed class JetStreamApiStats
{ {
/// <summary>Current API sampling level.</summary>
public int Level { get; set; } public int Level { get; set; }
/// <summary>Total JetStream API requests processed.</summary>
public ulong Total { get; set; } public ulong Total { get; set; }
/// <summary>Total JetStream API requests that returned an error.</summary>
public ulong Errors { get; set; } public ulong Errors { get; set; }
/// <summary>Number of API requests currently in flight.</summary>
public int Inflight { get; set; } public int Inflight { get; set; }
} }
@@ -18,10 +22,15 @@ public sealed class JetStreamApiStats
/// </summary> /// </summary>
public sealed class JetStreamTier public sealed class JetStreamTier
{ {
/// <summary>Name of the storage tier.</summary>
public string Name { get; set; } = string.Empty; public string Name { get; set; } = string.Empty;
/// <summary>Current memory usage in bytes for this tier.</summary>
public long Memory { get; set; } public long Memory { get; set; }
/// <summary>Current file-store usage in bytes for this tier.</summary>
public long Store { get; set; } public long Store { get; set; }
/// <summary>Number of streams in this tier.</summary>
public int Streams { get; set; } public int Streams { get; set; }
/// <summary>Number of consumers in this tier.</summary>
public int Consumers { get; set; } public int Consumers { get; set; }
} }
@@ -31,14 +40,23 @@ public sealed class JetStreamTier
/// </summary> /// </summary>
public sealed class JetStreamAccountLimits public sealed class JetStreamAccountLimits
{ {
/// <summary>Maximum memory bytes allowed for this account.</summary>
public long MaxMemory { get; set; } public long MaxMemory { get; set; }
/// <summary>Maximum file-store bytes allowed for this account.</summary>
public long MaxStore { get; set; } public long MaxStore { get; set; }
/// <summary>Maximum number of streams allowed for this account.</summary>
public int MaxStreams { get; set; } public int MaxStreams { get; set; }
/// <summary>Maximum number of consumers allowed for this account.</summary>
public int MaxConsumers { get; set; } public int MaxConsumers { get; set; }
/// <summary>Maximum unacknowledged messages allowed across consumers.</summary>
public int MaxAckPending { get; set; } public int MaxAckPending { get; set; }
/// <summary>Maximum bytes per memory-backed stream.</summary>
public long MemoryMaxStreamBytes { get; set; } public long MemoryMaxStreamBytes { get; set; }
/// <summary>Maximum bytes per file-backed stream.</summary>
public long StoreMaxStreamBytes { get; set; } public long StoreMaxStreamBytes { get; set; }
/// <summary>Indicates whether explicit max byte limits are required.</summary>
public bool MaxBytesRequired { get; set; } public bool MaxBytesRequired { get; set; }
/// <summary>Per-tier limits and usage details keyed by tier name.</summary>
public Dictionary<string, JetStreamTier> Tiers { get; set; } = new(StringComparer.Ordinal); public Dictionary<string, JetStreamTier> Tiers { get; set; } = new(StringComparer.Ordinal);
} }
@@ -48,11 +66,18 @@ public sealed class JetStreamAccountLimits
/// </summary> /// </summary>
public sealed class JetStreamStats public sealed class JetStreamStats
{ {
/// <summary>Total memory bytes currently used by JetStream.</summary>
public long Memory { get; set; } public long Memory { get; set; }
/// <summary>Total file-store bytes currently used by JetStream.</summary>
public long Store { get; set; } public long Store { get; set; }
/// <summary>Memory bytes reserved by configured account limits.</summary>
public long ReservedMemory { get; set; } public long ReservedMemory { get; set; }
/// <summary>File-store bytes reserved by configured account limits.</summary>
public long ReservedStore { get; set; } public long ReservedStore { get; set; }
/// <summary>Number of accounts with JetStream enabled.</summary>
public int Accounts { get; set; } public int Accounts { get; set; }
/// <summary>Number of high-availability assets under JetStream management.</summary>
public int HaAssets { get; set; } public int HaAssets { get; set; }
/// <summary>JetStream API usage counters.</summary>
public JetStreamApiStats Api { get; set; } = new(); public JetStreamApiStats Api { get; set; } = new();
} }
@@ -96,6 +96,10 @@ public sealed class MirrorCoordinator : IAsyncDisposable
// Error state tracking // Error state tracking
private string? _errorMessage; private string? _errorMessage;
/// <summary>
/// Initializes a mirror coordinator that applies origin stream messages to a local target store.
/// </summary>
/// <param name="targetStore">Local stream store that persists mirrored messages.</param>
public MirrorCoordinator(IStreamStore targetStore) public MirrorCoordinator(IStreamStore targetStore)
{ {
_targetStore = targetStore; _targetStore = targetStore;
@@ -111,6 +115,8 @@ public sealed class MirrorCoordinator : IAsyncDisposable
/// This is the direct-call path used when the origin and mirror are in the same process. /// This is the direct-call path used when the origin and mirror are in the same process.
/// Go reference: server/stream.go:2863-3014 (processInboundMirrorMsg) /// Go reference: server/stream.go:2863-3014 (processInboundMirrorMsg)
/// </summary> /// </summary>
/// <param name="message">Origin stream message to replicate into the mirror stream.</param>
/// <param name="ct">Cancellation token used to abort replication during shutdown.</param>
public async Task OnOriginAppendAsync(StoredMessage message, CancellationToken ct) public async Task OnOriginAppendAsync(StoredMessage message, CancellationToken ct)
{ {
// Go: sseq == mset.mirror.sseq+1 — normal in-order delivery // Go: sseq == mset.mirror.sseq+1 — normal in-order delivery
@@ -135,6 +141,7 @@ public sealed class MirrorCoordinator : IAsyncDisposable
/// Enqueues a message for processing by the background sync loop. /// Enqueues a message for processing by the background sync loop.
/// Used when messages arrive asynchronously (e.g., from a pull consumer on the origin). /// Used when messages arrive asynchronously (e.g., from a pull consumer on the origin).
/// </summary> /// </summary>
/// <param name="message">Origin message queued for asynchronous mirror processing.</param>
public bool TryEnqueue(StoredMessage message) public bool TryEnqueue(StoredMessage message)
{ {
return _inbound.Writer.TryWrite(message); return _inbound.Writer.TryWrite(message);
@@ -163,6 +170,8 @@ public sealed class MirrorCoordinator : IAsyncDisposable
/// actively pulls batches from the origin. /// actively pulls batches from the origin.
/// Go reference: server/stream.go:3125-3400 (setupMirrorConsumer) /// Go reference: server/stream.go:3125-3400 (setupMirrorConsumer)
/// </summary> /// </summary>
/// <param name="originStore">Origin store queried for mirror catch-up and incremental sync.</param>
/// <param name="batchSize">Maximum messages applied per pull iteration.</param>
public void StartPullSyncLoop(IStreamStore originStore, int batchSize = DefaultBatchSize) public void StartPullSyncLoop(IStreamStore originStore, int batchSize = DefaultBatchSize)
{ {
lock (_gate) lock (_gate)
@@ -254,6 +263,7 @@ public sealed class MirrorCoordinator : IAsyncDisposable
/// Records the next received sequence number from the origin stream. /// Records the next received sequence number from the origin stream.
/// Sets gap state when a gap (skipped sequences) is detected. /// Sets gap state when a gap (skipped sequences) is detected.
/// </summary> /// </summary>
/// <param name="seq">Observed origin sequence number.</param>
public void RecordSourceSeq(ulong seq) public void RecordSourceSeq(ulong seq)
{ {
if (_expectedOriginSeq > 0 && seq > _expectedOriginSeq + 1) if (_expectedOriginSeq > 0 && seq > _expectedOriginSeq + 1)
@@ -270,6 +280,7 @@ public sealed class MirrorCoordinator : IAsyncDisposable
// ------------------------------------------------------------------------- // -------------------------------------------------------------------------
/// <summary>Sets the coordinator into an error state with the given message.</summary> /// <summary>Sets the coordinator into an error state with the given message.</summary>
/// <param name="message">Human-readable mirror synchronization failure reason.</param>
public void SetError(string message) => _errorMessage = message; public void SetError(string message) => _errorMessage = message;
/// <summary>Clears the error state.</summary> /// <summary>Clears the error state.</summary>
@@ -279,6 +290,7 @@ public sealed class MirrorCoordinator : IAsyncDisposable
/// Reports current health state for monitoring. /// Reports current health state for monitoring.
/// Go reference: server/stream.go:2739-2743 (mirrorInfo), 2698-2736 (sourceInfo) /// Go reference: server/stream.go:2739-2743 (mirrorInfo), 2698-2736 (sourceInfo)
/// </summary> /// </summary>
/// <param name="originLastSeq">Optional latest sequence from the origin stream for lag calculation.</param>
public MirrorHealthReport GetHealthReport(ulong? originLastSeq = null) public MirrorHealthReport GetHealthReport(ulong? originLastSeq = null)
{ {
var lag = originLastSeq.HasValue && originLastSeq.Value > LastOriginSequence var lag = originLastSeq.HasValue && originLastSeq.Value > LastOriginSequence
@@ -301,6 +313,7 @@ public sealed class MirrorCoordinator : IAsyncDisposable
/// Returns a structured monitoring response for this mirror. /// Returns a structured monitoring response for this mirror.
/// Go reference: server/stream.go:2739-2743 (mirrorInfo building StreamSourceInfo) /// Go reference: server/stream.go:2739-2743 (mirrorInfo building StreamSourceInfo)
/// </summary> /// </summary>
/// <param name="streamName">Local mirror stream name exposed in monitoring responses.</param>
public MirrorInfoResponse GetMirrorInfo(string streamName) public MirrorInfoResponse GetMirrorInfo(string streamName)
{ {
var report = GetHealthReport(); var report = GetHealthReport();
@@ -313,6 +326,9 @@ public sealed class MirrorCoordinator : IAsyncDisposable
}; };
} }
/// <summary>
/// Stops mirror synchronization and completes inbound processing resources.
/// </summary>
public async ValueTask DisposeAsync() public async ValueTask DisposeAsync()
{ {
await StopAsync(); await StopAsync();
@@ -471,11 +487,17 @@ public sealed class MirrorCoordinator : IAsyncDisposable
/// </summary> /// </summary>
public sealed record MirrorHealthReport public sealed record MirrorHealthReport
{ {
/// <summary>Last origin stream sequence that has been persisted locally.</summary>
public ulong LastOriginSequence { get; init; } public ulong LastOriginSequence { get; init; }
/// <summary>UTC timestamp of the last successful mirror apply.</summary>
public DateTime LastSyncUtc { get; init; } public DateTime LastSyncUtc { get; init; }
/// <summary>Difference between origin head sequence and mirrored sequence.</summary>
public ulong Lag { get; init; } public ulong Lag { get; init; }
/// <summary>Count of consecutive synchronization failures.</summary>
public int ConsecutiveFailures { get; init; } public int ConsecutiveFailures { get; init; }
/// <summary>Whether the mirror sync loop is currently active.</summary>
public bool IsRunning { get; init; } public bool IsRunning { get; init; }
/// <summary>Whether sync activity appears stale based on heartbeat interval.</summary>
public bool IsStalled { get; init; } public bool IsStalled { get; init; }
} }
@@ -102,6 +102,15 @@ public sealed class SourceCoordinator : IAsyncDisposable
// Used for delta computation during aggregation. // Used for delta computation during aggregation.
private readonly Dictionary<string, long> _sourceCounterValues = new(StringComparer.Ordinal); private readonly Dictionary<string, long> _sourceCounterValues = new(StringComparer.Ordinal);
/// <summary>
/// Initializes a coordinator that mirrors one source stream into the target stream store.
/// </summary>
/// <param name="targetStore">
/// Target stream persistence used to store mirrored messages and maintain target-side ordering.
/// </param>
/// <param name="sourceConfig">
/// Source stream replication contract, including filter, account, transform, and dedup settings.
/// </param>
public SourceCoordinator(IStreamStore targetStore, StreamSourceConfig sourceConfig) public SourceCoordinator(IStreamStore targetStore, StreamSourceConfig sourceConfig)
{ {
_targetStore = targetStore; _targetStore = targetStore;
@@ -159,6 +168,8 @@ public sealed class SourceCoordinator : IAsyncDisposable
/// This is the direct-call path used when the origin and target are in the same process. /// This is the direct-call path used when the origin and target are in the same process.
/// Go reference: server/stream.go:3860-4007 (processInboundSourceMsg) /// Go reference: server/stream.go:3860-4007 (processInboundSourceMsg)
/// </summary> /// </summary>
/// <param name="message">Source stream message candidate to replicate into the target stream.</param>
/// <param name="ct">Cancellation token that stops replication when mirror coordination is shutting down.</param>
public async Task OnOriginAppendAsync(StoredMessage message, CancellationToken ct) public async Task OnOriginAppendAsync(StoredMessage message, CancellationToken ct)
{ {
// Account isolation: skip messages from different accounts. // Account isolation: skip messages from different accounts.
@@ -223,6 +234,7 @@ public sealed class SourceCoordinator : IAsyncDisposable
/// <summary> /// <summary>
/// Enqueues a message for processing by the background sync loop. /// Enqueues a message for processing by the background sync loop.
/// </summary> /// </summary>
/// <param name="message">Source message queued for asynchronous mirror processing.</param>
public bool TryEnqueue(StoredMessage message) public bool TryEnqueue(StoredMessage message)
{ {
return _inbound.Writer.TryWrite(message); return _inbound.Writer.TryWrite(message);
@@ -248,6 +260,8 @@ public sealed class SourceCoordinator : IAsyncDisposable
/// Starts a pull-based sync loop that actively fetches from the origin store. /// Starts a pull-based sync loop that actively fetches from the origin store.
/// Go reference: server/stream.go:3474-3720 (setupSourceConsumer + trySetupSourceConsumer) /// Go reference: server/stream.go:3474-3720 (setupSourceConsumer + trySetupSourceConsumer)
/// </summary> /// </summary>
/// <param name="originStore">Origin store used as the authoritative source for backfill and catch-up reads.</param>
/// <param name="batchSize">Maximum number of origin messages processed per pull iteration.</param>
public void StartPullSyncLoop(IStreamStore originStore, int batchSize = DefaultBatchSize) public void StartPullSyncLoop(IStreamStore originStore, int batchSize = DefaultBatchSize)
{ {
lock (_gate) lock (_gate)
@@ -296,6 +310,7 @@ public sealed class SourceCoordinator : IAsyncDisposable
/// Reports current health state for monitoring. /// Reports current health state for monitoring.
/// Go reference: server/stream.go:2687-2695 (sourcesInfo) /// Go reference: server/stream.go:2687-2695 (sourcesInfo)
/// </summary> /// </summary>
/// <param name="originLastSeq">Optional current origin sequence used to compute real-time lag from monitoring data.</param>
public SourceHealthReport GetHealthReport(ulong? originLastSeq = null) public SourceHealthReport GetHealthReport(ulong? originLastSeq = null)
{ {
var lag = originLastSeq.HasValue && originLastSeq.Value > LastOriginSequence var lag = originLastSeq.HasValue && originLastSeq.Value > LastOriginSequence
@@ -335,6 +350,9 @@ public sealed class SourceCoordinator : IAsyncDisposable
}; };
} }
/// <summary>
/// Stops source synchronization and completes the inbound queue used by the mirror worker.
/// </summary>
public async ValueTask DisposeAsync() public async ValueTask DisposeAsync()
{ {
await StopAsync(); await StopAsync();
@@ -576,6 +594,7 @@ public sealed class SourceCoordinator : IAsyncDisposable
/// Returns true if the given message ID is already present in the dedup window. /// Returns true if the given message ID is already present in the dedup window.
/// Go reference: server/stream.go duplicate window check /// Go reference: server/stream.go duplicate window check
/// </summary> /// </summary>
/// <param name="msgId">Client-provided Nats-Msg-Id used to enforce at-most-once mirror replay.</param>
public bool IsDuplicate(string msgId) public bool IsDuplicate(string msgId)
{ {
PruneDedupWindowIfNeeded(); PruneDedupWindowIfNeeded();
@@ -586,6 +605,7 @@ public sealed class SourceCoordinator : IAsyncDisposable
/// Records a message ID in the dedup window with the current timestamp. /// Records a message ID in the dedup window with the current timestamp.
/// Go reference: server/stream.go duplicate window tracking /// Go reference: server/stream.go duplicate window tracking
/// </summary> /// </summary>
/// <param name="msgId">Client-provided Nats-Msg-Id to mark as recently observed for this source.</param>
public void RecordMsgId(string msgId) public void RecordMsgId(string msgId)
{ {
_dedupWindow[msgId] = DateTime.UtcNow; _dedupWindow[msgId] = DateTime.UtcNow;
@@ -597,6 +617,7 @@ public sealed class SourceCoordinator : IAsyncDisposable
/// time-based pruning done by <see cref="PruneDedupWindowIfNeeded"/>. /// time-based pruning done by <see cref="PruneDedupWindowIfNeeded"/>.
/// Go reference: server/stream.go duplicate window pruning /// Go reference: server/stream.go duplicate window pruning
/// </summary> /// </summary>
/// <param name="cutoff">UTC threshold; entries older than this instant are removed from dedup state.</param>
public void PruneDedupWindow(DateTimeOffset cutoff) public void PruneDedupWindow(DateTimeOffset cutoff)
{ {
var cutoffDt = cutoff.UtcDateTime; var cutoffDt = cutoff.UtcDateTime;
@@ -642,15 +663,25 @@ public sealed class SourceCoordinator : IAsyncDisposable
/// </summary> /// </summary>
public sealed record SourceHealthReport public sealed record SourceHealthReport
{ {
/// <summary>Name of the origin stream being mirrored.</summary>
public string SourceName { get; init; } = string.Empty; public string SourceName { get; init; } = string.Empty;
/// <summary>Optional filter that restricts which source subjects are mirrored.</summary>
public string? FilterSubject { get; init; } public string? FilterSubject { get; init; }
/// <summary>Last origin sequence number that was persisted to the target stream.</summary>
public ulong LastOriginSequence { get; init; } public ulong LastOriginSequence { get; init; }
/// <summary>Timestamp of the most recent successful mirror write.</summary>
public DateTime LastSyncUtc { get; init; } public DateTime LastSyncUtc { get; init; }
/// <summary>Difference between latest known origin sequence and last replicated sequence.</summary>
public ulong Lag { get; init; } public ulong Lag { get; init; }
/// <summary>Count of consecutive sync failures since the last successful mirror operation.</summary>
public int ConsecutiveFailures { get; init; } public int ConsecutiveFailures { get; init; }
/// <summary>Whether the coordinator currently has an active sync loop.</summary>
public bool IsRunning { get; init; } public bool IsRunning { get; init; }
/// <summary>Whether the source is considered stalled based on heartbeat and last-sync time.</summary>
public bool IsStalled { get; init; } public bool IsStalled { get; init; }
/// <summary>Total number of source messages skipped because they did not match the configured filter.</summary>
public long FilteredOutCount { get; init; } public long FilteredOutCount { get; init; }
/// <summary>Total number of source messages skipped because their message IDs were already seen.</summary>
public long DeduplicatedCount { get; init; } public long DeduplicatedCount { get; init; }
} }
@@ -30,9 +30,13 @@ public sealed record SnapshotRestoreResult(
file sealed class TarMessageEntry file sealed class TarMessageEntry
{ {
/// <summary>Original stream sequence used to restore message order.</summary>
public ulong Sequence { get; init; } public ulong Sequence { get; init; }
/// <summary>Subject that the message was originally stored under.</summary>
public string Subject { get; init; } = string.Empty; public string Subject { get; init; } = string.Empty;
/// <summary>Base64-encoded payload bytes stored in the snapshot archive.</summary>
public string Payload { get; init; } = string.Empty; // base-64 public string Payload { get; init; } = string.Empty; // base-64
/// <summary>UTC timestamp encoded using ISO-8601 format.</summary>
public string Timestamp { get; init; } = string.Empty; // ISO-8601 UTC public string Timestamp { get; init; } = string.Empty; // ISO-8601 UTC
} }
@@ -46,9 +50,20 @@ public sealed class StreamSnapshotService
// Existing thin wrappers (kept for API compatibility) // Existing thin wrappers (kept for API compatibility)
// ────────────────────────────────────────────────────────────────────── // ──────────────────────────────────────────────────────────────────────
/// <summary>
/// Creates a store-native snapshot through the stream store implementation.
/// </summary>
/// <param name="stream">Stream whose persistent state is being snapshotted.</param>
/// <param name="ct">Cancellation token for snapshot creation.</param>
public ValueTask<byte[]> SnapshotAsync(StreamHandle stream, CancellationToken ct) public ValueTask<byte[]> SnapshotAsync(StreamHandle stream, CancellationToken ct)
=> stream.Store.CreateSnapshotAsync(ct); => stream.Store.CreateSnapshotAsync(ct);
/// <summary>
/// Restores a store-native snapshot through the stream store implementation.
/// </summary>
/// <param name="stream">Stream receiving restored state.</param>
/// <param name="snapshot">Snapshot payload produced by the underlying store format.</param>
/// <param name="ct">Cancellation token for restore execution.</param>
public ValueTask RestoreAsync(StreamHandle stream, ReadOnlyMemory<byte> snapshot, CancellationToken ct) public ValueTask RestoreAsync(StreamHandle stream, ReadOnlyMemory<byte> snapshot, CancellationToken ct)
=> stream.Store.RestoreSnapshotAsync(snapshot, ct); => stream.Store.RestoreSnapshotAsync(snapshot, ct);
@@ -66,6 +81,8 @@ public sealed class StreamSnapshotService
/// messages/000002.json /// messages/000002.json
/// … /// …
/// </summary> /// </summary>
/// <param name="stream">Stream to snapshot, including config and persisted messages.</param>
/// <param name="ct">Cancellation token for snapshot generation.</param>
public async Task<byte[]> CreateTarSnapshotAsync(StreamHandle stream, CancellationToken ct) public async Task<byte[]> CreateTarSnapshotAsync(StreamHandle stream, CancellationToken ct)
{ {
// Collect messages first (outside the TAR buffer so we hold no lock). // Collect messages first (outside the TAR buffer so we hold no lock).
@@ -103,6 +120,9 @@ public sealed class StreamSnapshotService
/// Decompress a Snappy-compressed TAR archive, validate stream.json, and /// Decompress a Snappy-compressed TAR archive, validate stream.json, and
/// replay all message entries back into the store. /// replay all message entries back into the store.
/// </summary> /// </summary>
/// <param name="stream">Target stream that receives restored messages.</param>
/// <param name="snapshot">Snappy-compressed TAR snapshot bytes.</param>
/// <param name="ct">Cancellation token for restore execution.</param>
public async Task<SnapshotRestoreResult> RestoreTarSnapshotAsync( public async Task<SnapshotRestoreResult> RestoreTarSnapshotAsync(
StreamHandle stream, StreamHandle stream,
ReadOnlyMemory<byte> snapshot, ReadOnlyMemory<byte> snapshot,
@@ -174,6 +194,9 @@ public sealed class StreamSnapshotService
/// Same as <see cref="CreateTarSnapshotAsync"/> but cancels automatically if /// Same as <see cref="CreateTarSnapshotAsync"/> but cancels automatically if
/// the operation has not completed within <paramref name="deadline"/>. /// the operation has not completed within <paramref name="deadline"/>.
/// </summary> /// </summary>
/// <param name="stream">Stream to snapshot.</param>
/// <param name="deadline">Maximum allowed runtime before cancellation.</param>
/// <param name="ct">Caller cancellation token linked with the deadline token.</param>
public async Task<byte[]> CreateTarSnapshotWithDeadlineAsync( public async Task<byte[]> CreateTarSnapshotWithDeadlineAsync(
StreamHandle stream, StreamHandle stream,
TimeSpan deadline, TimeSpan deadline,
@@ -39,6 +39,11 @@ public sealed class ConsumerFileStore : IConsumerStore
// Reference: golang/nats-server/server/errors.go // Reference: golang/nats-server/server/errors.go
public static readonly Exception ErrNoAckPolicy = new InvalidOperationException("ErrNoAckPolicy"); public static readonly Exception ErrNoAckPolicy = new InvalidOperationException("ErrNoAckPolicy");
/// <summary>
/// Creates a file-backed consumer state store and starts background flush processing.
/// </summary>
/// <param name="stateFile">Path to the on-disk consumer state file.</param>
/// <param name="cfg">Consumer configuration that drives ack and redelivery persistence behavior.</param>
public ConsumerFileStore(string stateFile, ConsumerConfig cfg) public ConsumerFileStore(string stateFile, ConsumerConfig cfg)
{ {
_stateFile = stateFile; _stateFile = stateFile;
@@ -63,6 +68,7 @@ public sealed class ConsumerFileStore : IConsumerStore
} }
// Go: consumerFileStore.SetStarting — filestore.go:11660 // Go: consumerFileStore.SetStarting — filestore.go:11660
/// <inheritdoc />
public void SetStarting(ulong sseq) public void SetStarting(ulong sseq)
{ {
lock (_mu) lock (_mu)
@@ -72,6 +78,7 @@ public sealed class ConsumerFileStore : IConsumerStore
} }
// Go: consumerFileStore.UpdateStarting — filestore.go:11665 // Go: consumerFileStore.UpdateStarting — filestore.go:11665
/// <inheritdoc />
public void UpdateStarting(ulong sseq) public void UpdateStarting(ulong sseq)
{ {
lock (_mu) lock (_mu)
@@ -81,6 +88,7 @@ public sealed class ConsumerFileStore : IConsumerStore
} }
// Go: consumerFileStore.Reset — filestore.go:11670 // Go: consumerFileStore.Reset — filestore.go:11670
/// <inheritdoc />
public void Reset(ulong sseq) public void Reset(ulong sseq)
{ {
lock (_mu) lock (_mu)
@@ -94,6 +102,7 @@ public sealed class ConsumerFileStore : IConsumerStore
} }
// Go: consumerFileStore.HasState — filestore.go // Go: consumerFileStore.HasState — filestore.go
/// <inheritdoc />
public bool HasState() public bool HasState()
{ {
lock (_mu) lock (_mu)
@@ -102,6 +111,7 @@ public sealed class ConsumerFileStore : IConsumerStore
// Go: consumerFileStore.UpdateDelivered — filestore.go:11700 // Go: consumerFileStore.UpdateDelivered — filestore.go:11700
// dseq=consumer delivery seq, sseq=stream seq, dc=delivery count, ts=Unix nanosec timestamp // dseq=consumer delivery seq, sseq=stream seq, dc=delivery count, ts=Unix nanosec timestamp
/// <inheritdoc />
public void UpdateDelivered(ulong dseq, ulong sseq, ulong dc, long ts) public void UpdateDelivered(ulong dseq, ulong sseq, ulong dc, long ts)
{ {
lock (_mu) lock (_mu)
@@ -138,6 +148,7 @@ public sealed class ConsumerFileStore : IConsumerStore
} }
// Go: consumerFileStore.UpdateAcks — filestore.go:11760 // Go: consumerFileStore.UpdateAcks — filestore.go:11760
/// <inheritdoc />
public void UpdateAcks(ulong dseq, ulong sseq) public void UpdateAcks(ulong dseq, ulong sseq)
{ {
lock (_mu) lock (_mu)
@@ -171,6 +182,7 @@ public sealed class ConsumerFileStore : IConsumerStore
} }
// Go: consumerFileStore.Update — filestore.go // Go: consumerFileStore.Update — filestore.go
/// <inheritdoc />
public void Update(ConsumerState state) public void Update(ConsumerState state)
{ {
lock (_mu) lock (_mu)
@@ -181,6 +193,7 @@ public sealed class ConsumerFileStore : IConsumerStore
} }
// Go: consumerFileStore.State — filestore.go:12103 // Go: consumerFileStore.State — filestore.go:12103
/// <inheritdoc />
public ConsumerState State() public ConsumerState State()
{ {
lock (_mu) lock (_mu)
@@ -207,12 +220,14 @@ public sealed class ConsumerFileStore : IConsumerStore
} }
// Go: consumerFileStore.BorrowState — filestore.go:12109 // Go: consumerFileStore.BorrowState — filestore.go:12109
/// <inheritdoc />
public ConsumerState BorrowState() public ConsumerState BorrowState()
{ {
lock (_mu) return _state; lock (_mu) return _state;
} }
// Go: consumerFileStore.EncodedState — filestore.go // Go: consumerFileStore.EncodedState — filestore.go
/// <inheritdoc />
public byte[] EncodedState() public byte[] EncodedState()
{ {
lock (_mu) lock (_mu)
@@ -220,9 +235,11 @@ public sealed class ConsumerFileStore : IConsumerStore
} }
// Go: consumerFileStore.Type — filestore.go:12099 // Go: consumerFileStore.Type — filestore.go:12099
/// <inheritdoc />
public StorageType Type() => StorageType.File; public StorageType Type() => StorageType.File;
// Go: consumerFileStore.Stop — filestore.go:12327 // Go: consumerFileStore.Stop — filestore.go:12327
/// <inheritdoc />
public void Stop() public void Stop()
{ {
lock (_mu) lock (_mu)
@@ -240,6 +257,7 @@ public sealed class ConsumerFileStore : IConsumerStore
} }
// Go: consumerFileStore.Delete — filestore.go:12382 // Go: consumerFileStore.Delete — filestore.go:12382
/// <inheritdoc />
public void Delete() public void Delete()
{ {
lock (_mu) lock (_mu)
@@ -258,6 +276,7 @@ public sealed class ConsumerFileStore : IConsumerStore
} }
// Go: consumerFileStore.StreamDelete — filestore.go:12387 // Go: consumerFileStore.StreamDelete — filestore.go:12387
/// <inheritdoc />
public void StreamDelete() public void StreamDelete()
{ {
Delete(); Delete();
+214 -5
View File
@@ -91,6 +91,12 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
private const int CoalesceMinimum = 16 * 1024; // 16KB — Go: filestore.go:328 private const int CoalesceMinimum = 16 * 1024; // 16KB — Go: filestore.go:328
private const int MaxFlushWaitMs = 8; // 8ms — Go: filestore.go:331 private const int MaxFlushWaitMs = 8; // 8ms — Go: filestore.go:331
// Go: msgBlock.needSync — deferred fsync flag. Set when a block has unflushed
// data that needs to reach stable storage. The background flush loop handles
// the actual fsync outside the hot path, matching Go's behaviour.
// Reference: golang/nats-server/server/filestore.go:258 (needSync field).
private readonly ConcurrentQueue<MsgBlock> _needSyncBlocks = new();
// Go: filestore.go — generation counter for cache invalidation. // Go: filestore.go — generation counter for cache invalidation.
// Incremented on every write (Append/StoreRawMsg) and delete (Remove/Purge/Compact). // Incremented on every write (Append/StoreRawMsg) and delete (Remove/Purge/Compact).
// NumFiltered caches results keyed by (filter, generation) so repeated calls for // NumFiltered caches results keyed by (filter, generation) so repeated calls for
@@ -99,15 +105,39 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
private ulong _generation; private ulong _generation;
private readonly Dictionary<string, (ulong Generation, ulong Count)> _numFilteredCache = new(StringComparer.Ordinal); private readonly Dictionary<string, (ulong Generation, ulong Count)> _numFilteredCache = new(StringComparer.Ordinal);
/// <summary>
/// Number of active message blocks currently managed by the store.
/// </summary>
public int BlockCount => _blocks.Count; public int BlockCount => _blocks.Count;
/// <summary>
/// Indicates whether recovery used the index manifest fast path at startup.
/// </summary>
public bool UsedIndexManifestOnStartup { get; private set; } public bool UsedIndexManifestOnStartup { get; private set; }
// IStreamStore cached state properties — O(1), maintained incrementally. // IStreamStore cached state properties — O(1), maintained incrementally.
/// <summary>
/// Highest sequence watermark observed by this stream store.
/// </summary>
public ulong LastSeq => _last; public ulong LastSeq => _last;
/// <summary>
/// Current number of live messages tracked by the stream.
/// </summary>
public ulong MessageCount => _messageCount; public ulong MessageCount => _messageCount;
/// <summary>
/// Total bytes of live message payload and headers tracked by the stream.
/// </summary>
public ulong TotalBytes => _totalBytes; public ulong TotalBytes => _totalBytes;
/// <inheritdoc />
ulong IStreamStore.FirstSeq => _messageCount == 0 ? (_first > 0 ? _first : 0UL) : _firstSeq; ulong IStreamStore.FirstSeq => _messageCount == 0 ? (_first > 0 ? _first : 0UL) : _firstSeq;
/// <summary>
/// Creates a file-backed JetStream store and recovers persisted stream blocks.
/// </summary>
/// <param name="options">File store options controlling directory layout, limits, and transforms.</param>
public FileStore(FileStoreOptions options) public FileStore(FileStoreOptions options)
{ {
_options = options; _options = options;
@@ -136,6 +166,12 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
_flushTask = Task.Run(() => FlushLoopAsync(_flushCts.Token)); _flushTask = Task.Run(() => FlushLoopAsync(_flushCts.Token));
} }
/// <summary>
/// Appends a new message to the stream and returns the assigned sequence.
/// </summary>
/// <param name="subject">Publish subject for the appended message.</param>
/// <param name="payload">Message payload bytes.</param>
/// <param name="ct">Cancellation token for async callers.</param>
public ValueTask<ulong> AppendAsync(string subject, ReadOnlyMemory<byte> payload, CancellationToken ct) public ValueTask<ulong> AppendAsync(string subject, ReadOnlyMemory<byte> payload, CancellationToken ct)
{ {
if (_stopped) if (_stopped)
@@ -198,11 +234,21 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
return ValueTask.FromResult(_last); return ValueTask.FromResult(_last);
} }
/// <summary>
/// Loads a stored message by exact sequence.
/// </summary>
/// <param name="sequence">Sequence to load.</param>
/// <param name="ct">Cancellation token for async callers.</param>
public ValueTask<StoredMessage?> LoadAsync(ulong sequence, CancellationToken ct) public ValueTask<StoredMessage?> LoadAsync(ulong sequence, CancellationToken ct)
{ {
return ValueTask.FromResult(MaterializeMessage(sequence)); return ValueTask.FromResult(MaterializeMessage(sequence));
} }
/// <summary>
/// Loads the most recent message for a subject.
/// </summary>
/// <param name="subject">Subject to query.</param>
/// <param name="ct">Cancellation token for async callers.</param>
public ValueTask<StoredMessage?> LoadLastBySubjectAsync(string subject, CancellationToken ct) public ValueTask<StoredMessage?> LoadLastBySubjectAsync(string subject, CancellationToken ct)
{ {
if (_lastSequenceBySubject.TryGetValue(subject, out var sequence)) if (_lastSequenceBySubject.TryGetValue(subject, out var sequence))
@@ -214,6 +260,10 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
return ValueTask.FromResult<StoredMessage?>(null); return ValueTask.FromResult<StoredMessage?>(null);
} }
/// <summary>
/// Lists all live messages in sequence order.
/// </summary>
/// <param name="ct">Cancellation token for async callers.</param>
public ValueTask<IReadOnlyList<StoredMessage>> ListAsync(CancellationToken ct) public ValueTask<IReadOnlyList<StoredMessage>> ListAsync(CancellationToken ct)
{ {
var messages = _meta.Keys var messages = _meta.Keys
@@ -225,6 +275,11 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
return ValueTask.FromResult<IReadOnlyList<StoredMessage>>(messages); return ValueTask.FromResult<IReadOnlyList<StoredMessage>>(messages);
} }
/// <summary>
/// Removes a message by sequence when present.
/// </summary>
/// <param name="sequence">Sequence to remove.</param>
/// <param name="ct">Cancellation token for async callers.</param>
public ValueTask<bool> RemoveAsync(ulong sequence, CancellationToken ct) public ValueTask<bool> RemoveAsync(ulong sequence, CancellationToken ct)
{ {
if (!RemoveTrackedMessage(sequence, preserveHighWaterMark: false)) if (!RemoveTrackedMessage(sequence, preserveHighWaterMark: false))
@@ -238,6 +293,10 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
return ValueTask.FromResult(true); return ValueTask.FromResult(true);
} }
/// <summary>
/// Purges all messages and block files for this stream store.
/// </summary>
/// <param name="ct">Cancellation token for async callers.</param>
public ValueTask PurgeAsync(CancellationToken ct) public ValueTask PurgeAsync(CancellationToken ct)
{ {
// Stop the background flush loop before disposing blocks to prevent // Stop the background flush loop before disposing blocks to prevent
@@ -273,6 +332,10 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
return ValueTask.CompletedTask; return ValueTask.CompletedTask;
} }
/// <summary>
/// Creates a serialized snapshot of all live stream messages.
/// </summary>
/// <param name="ct">Cancellation token for async callers.</param>
public ValueTask<byte[]> CreateSnapshotAsync(CancellationToken ct) public ValueTask<byte[]> CreateSnapshotAsync(CancellationToken ct)
{ {
var snapshot = _meta.Keys var snapshot = _meta.Keys
@@ -295,6 +358,11 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
return ValueTask.FromResult(JsonSerializer.SerializeToUtf8Bytes(snapshot)); return ValueTask.FromResult(JsonSerializer.SerializeToUtf8Bytes(snapshot));
} }
/// <summary>
/// Restores stream messages from a previously created snapshot payload.
/// </summary>
/// <param name="snapshot">Serialized snapshot payload.</param>
/// <param name="ct">Cancellation token for async callers.</param>
public ValueTask RestoreSnapshotAsync(ReadOnlyMemory<byte> snapshot, CancellationToken ct) public ValueTask RestoreSnapshotAsync(ReadOnlyMemory<byte> snapshot, CancellationToken ct)
{ {
_meta.Clear(); _meta.Clear();
@@ -347,6 +415,10 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
return ValueTask.CompletedTask; return ValueTask.CompletedTask;
} }
/// <summary>
/// Returns lightweight stream counters used by API responses.
/// </summary>
/// <param name="ct">Cancellation token for async callers.</param>
public ValueTask<ApiStreamState> GetStateAsync(CancellationToken ct) public ValueTask<ApiStreamState> GetStateAsync(CancellationToken ct)
{ {
return ValueTask.FromResult(new ApiStreamState return ValueTask.FromResult(new ApiStreamState
@@ -358,6 +430,10 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
}); });
} }
/// <summary>
/// Trims oldest messages until live count is at or below the configured limit.
/// </summary>
/// <param name="maxMessages">Maximum live message count to retain.</param>
public void TrimToMaxMessages(ulong maxMessages) public void TrimToMaxMessages(ulong maxMessages)
{ {
var trimmed = false; var trimmed = false;
@@ -390,6 +466,10 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
/// this specific message; otherwise the stream's MaxAgeMs applies. /// this specific message; otherwise the stream's MaxAgeMs applies.
/// Reference: golang/nats-server/server/filestore.go:6790 (storeMsg). /// Reference: golang/nats-server/server/filestore.go:6790 (storeMsg).
/// </summary> /// </summary>
/// <param name="subject">Publish subject for the message.</param>
/// <param name="hdr">Optional protocol headers.</param>
/// <param name="msg">Message payload bytes.</param>
/// <param name="ttl">Optional per-message TTL in nanoseconds.</param>
public (ulong Seq, long Ts) StoreMsg(string subject, byte[]? hdr, byte[] msg, long ttl) public (ulong Seq, long Ts) StoreMsg(string subject, byte[]? hdr, byte[] msg, long ttl)
{ {
if (_stopped) if (_stopped)
@@ -467,6 +547,9 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
/// Returns the number of messages removed. /// Returns the number of messages removed.
/// Reference: golang/nats-server/server/filestore.go — PurgeEx. /// Reference: golang/nats-server/server/filestore.go — PurgeEx.
/// </summary> /// </summary>
/// <param name="subject">Subject filter; empty means all subjects.</param>
/// <param name="seq">Upper inclusive sequence boundary (0 means stream last sequence).</param>
/// <param name="keep">Number of newest matching messages to keep.</param>
public ulong PurgeEx(string subject, ulong seq, ulong keep) public ulong PurgeEx(string subject, ulong seq, ulong keep)
{ {
// Go parity: empty subject with keep=0 and seq=0 is a full purge. // Go parity: empty subject with keep=0 and seq=0 is a full purge.
@@ -529,6 +612,7 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
/// and returns the count removed. /// and returns the count removed.
/// Reference: golang/nats-server/server/filestore.go — Compact. /// Reference: golang/nats-server/server/filestore.go — Compact.
/// </summary> /// </summary>
/// <param name="seq">Exclusive upper bound for removed sequences.</param>
public ulong Compact(ulong seq) public ulong Compact(ulong seq)
{ {
if (seq == 0) if (seq == 0)
@@ -566,6 +650,7 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
/// and updates the last sequence pointer. /// and updates the last sequence pointer.
/// Reference: golang/nats-server/server/filestore.go — Truncate. /// Reference: golang/nats-server/server/filestore.go — Truncate.
/// </summary> /// </summary>
/// <param name="seq">Highest sequence to keep.</param>
public void Truncate(ulong seq) public void Truncate(ulong seq)
{ {
if (seq == 0) if (seq == 0)
@@ -609,6 +694,7 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
/// Returns <c>_last + 1</c> if no message exists at or after <paramref name="t"/>. /// Returns <c>_last + 1</c> if no message exists at or after <paramref name="t"/>.
/// Reference: golang/nats-server/server/filestore.go — GetSeqFromTime. /// Reference: golang/nats-server/server/filestore.go — GetSeqFromTime.
/// </summary> /// </summary>
/// <param name="t">UTC timestamp used as lookup lower bound.</param>
public ulong GetSeqFromTime(DateTime t) public ulong GetSeqFromTime(DateTime t)
{ {
var utc = t.Kind == DateTimeKind.Utc ? t : t.ToUniversalTime(); var utc = t.Kind == DateTimeKind.Utc ? t : t.ToUniversalTime();
@@ -635,6 +721,8 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
/// messages already cached in <c>_meta</c> are filtered directly. /// messages already cached in <c>_meta</c> are filtered directly.
/// Reference: golang/nats-server/server/filestore.go:3191 (FilteredState). /// Reference: golang/nats-server/server/filestore.go:3191 (FilteredState).
/// </summary> /// </summary>
/// <param name="seq">Starting sequence lower bound.</param>
/// <param name="subject">Optional subject filter.</param>
public SimpleState FilteredState(ulong seq, string subject) public SimpleState FilteredState(ulong seq, string subject)
{ {
// Fast path: binary-search to find the first block whose LastSequence >= seq, // Fast path: binary-search to find the first block whose LastSequence >= seq,
@@ -712,6 +800,8 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
/// sets, which is deferred. This method is the extension point for that optimization. /// sets, which is deferred. This method is the extension point for that optimization.
/// Reference: golang/nats-server/server/filestore.go (block-level subject tracking). /// Reference: golang/nats-server/server/filestore.go (block-level subject tracking).
/// </summary> /// </summary>
/// <param name="filter">Subject filter used by the scan.</param>
/// <param name="firstBlock">First block candidate considered by the scan.</param>
public static bool CheckSkipFirstBlock(string filter, MsgBlock firstBlock) public static bool CheckSkipFirstBlock(string filter, MsgBlock firstBlock)
{ {
// Without per-block subject metadata we cannot skip based on subject alone. // Without per-block subject metadata we cannot skip based on subject alone.
@@ -732,6 +822,7 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
/// <paramref name="filterSubject"/>. Supports NATS wildcard filters. /// <paramref name="filterSubject"/>. Supports NATS wildcard filters.
/// Reference: golang/nats-server/server/filestore.go — SubjectsState. /// Reference: golang/nats-server/server/filestore.go — SubjectsState.
/// </summary> /// </summary>
/// <param name="filterSubject">Optional subject filter with wildcard support.</param>
public Dictionary<string, SimpleState> SubjectsState(string filterSubject) public Dictionary<string, SimpleState> SubjectsState(string filterSubject)
{ {
var result = new Dictionary<string, SimpleState>(StringComparer.Ordinal); var result = new Dictionary<string, SimpleState>(StringComparer.Ordinal);
@@ -771,6 +862,7 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
/// <paramref name="filterSubject"/>. Supports NATS wildcard filters. /// <paramref name="filterSubject"/>. Supports NATS wildcard filters.
/// Reference: golang/nats-server/server/filestore.go — SubjectsTotals. /// Reference: golang/nats-server/server/filestore.go — SubjectsTotals.
/// </summary> /// </summary>
/// <param name="filterSubject">Optional subject filter with wildcard support.</param>
public Dictionary<string, ulong> SubjectsTotals(string filterSubject) public Dictionary<string, ulong> SubjectsTotals(string filterSubject)
{ {
var result = new Dictionary<string, ulong>(StringComparer.Ordinal); var result = new Dictionary<string, ulong>(StringComparer.Ordinal);
@@ -834,6 +926,7 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
/// <see cref="StreamState.Subjects"/> dictionary. /// <see cref="StreamState.Subjects"/> dictionary.
/// Reference: golang/nats-server/server/filestore.go — FastState. /// Reference: golang/nats-server/server/filestore.go — FastState.
/// </summary> /// </summary>
/// <param name="state">State object to populate.</param>
public void FastState(ref StreamState state) public void FastState(ref StreamState state)
{ {
state.Msgs = _messageCount; state.Msgs = _messageCount;
@@ -1063,6 +1156,7 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
/// An empty or null filter counts all messages. /// An empty or null filter counts all messages.
/// Reference: golang/nats-server/server/filestore.go — fss NumFiltered (subject-state cache). /// Reference: golang/nats-server/server/filestore.go — fss NumFiltered (subject-state cache).
/// </summary> /// </summary>
/// <param name="filter">Subject filter; null or empty counts all messages.</param>
public ulong NumFiltered(string filter) public ulong NumFiltered(string filter)
{ {
var key = filter ?? string.Empty; var key = filter ?? string.Empty;
@@ -1083,6 +1177,9 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
return count; return count;
} }
/// <summary>
/// Asynchronously disposes this file store and flushes pending buffered writes.
/// </summary>
public async ValueTask DisposeAsync() public async ValueTask DisposeAsync()
{ {
// Stop the background flush loop first to prevent it from accessing // Stop the background flush loop first to prevent it from accessing
@@ -1118,6 +1215,7 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
/// Stops the store and deletes all persisted data (blocks, index files). /// Stops the store and deletes all persisted data (blocks, index files).
/// Reference: golang/nats-server/server/filestore.go — fileStore.Delete. /// Reference: golang/nats-server/server/filestore.go — fileStore.Delete.
/// </summary> /// </summary>
/// <param name="inline">Reserved for parity with Go signature; currently ignored.</param>
public void Delete(bool inline = false) public void Delete(bool inline = false)
{ {
Stop(); Stop();
@@ -1154,11 +1252,14 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
// Flush any pending buffered writes before sealing the outgoing block. // Flush any pending buffered writes before sealing the outgoing block.
_activeBlock?.FlushPending(); _activeBlock?.FlushPending();
// Go: filestore.go:4499 (flushPendingMsgsLocked) — evict the outgoing block's // Evict from write cache manager (tracking only, no fsync).
// write cache via WriteCacheManager before rotating to the new block.
// WriteCacheManager.EvictBlock flushes to disk then clears the cache.
if (_activeBlock is not null) if (_activeBlock is not null)
_writeCache.EvictBlock(_activeBlock.BlockId); {
_writeCache.EvictBlockNoSync(_activeBlock.BlockId);
// Defer fsync to the background flush loop, matching Go's needSync pattern.
// Reference: golang/nats-server/server/filestore.go:7207-7266
_needSyncBlocks.Enqueue(_activeBlock);
}
// Clear the write cache on the outgoing active block — it is now sealed. // Clear the write cache on the outgoing active block — it is now sealed.
// This frees memory; future reads on sealed blocks go to disk. // This frees memory; future reads on sealed blocks go to disk.
@@ -1869,6 +1970,7 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
/// Returns <c>true</c> if the sequence existed and was removed. /// Returns <c>true</c> if the sequence existed and was removed.
/// Reference: golang/nats-server/server/filestore.go — RemoveMsg. /// Reference: golang/nats-server/server/filestore.go — RemoveMsg.
/// </summary> /// </summary>
/// <param name="seq">Sequence to remove.</param>
public bool RemoveMsg(ulong seq) public bool RemoveMsg(ulong seq)
{ {
if (!RemoveTrackedMessage(seq, preserveHighWaterMark: true)) if (!RemoveTrackedMessage(seq, preserveHighWaterMark: true))
@@ -1888,6 +1990,7 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
/// Returns <c>true</c> if the sequence existed and was erased. /// Returns <c>true</c> if the sequence existed and was erased.
/// Reference: golang/nats-server/server/filestore.go:5890 (eraseMsg). /// Reference: golang/nats-server/server/filestore.go:5890 (eraseMsg).
/// </summary> /// </summary>
/// <param name="seq">Sequence to erase.</param>
public bool EraseMsg(ulong seq) public bool EraseMsg(ulong seq)
{ {
if (!RemoveTrackedMessage(seq, preserveHighWaterMark: true)) if (!RemoveTrackedMessage(seq, preserveHighWaterMark: true))
@@ -1908,6 +2011,7 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
/// Returns the skipped sequence number. /// Returns the skipped sequence number.
/// Reference: golang/nats-server/server/filestore.go — SkipMsg. /// Reference: golang/nats-server/server/filestore.go — SkipMsg.
/// </summary> /// </summary>
/// <param name="seq">Requested sequence or 0 to auto-assign next sequence.</param>
public ulong SkipMsg(ulong seq) public ulong SkipMsg(ulong seq)
{ {
// When seq is 0, auto-assign next sequence. // When seq is 0, auto-assign next sequence.
@@ -1946,6 +2050,8 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
/// (_last + 1); otherwise an <see cref="InvalidOperationException"/> is thrown /// (_last + 1); otherwise an <see cref="InvalidOperationException"/> is thrown
/// (Go: ErrSequenceMismatch). /// (Go: ErrSequenceMismatch).
/// </summary> /// </summary>
/// <param name="seq">Start sequence or 0 to use next available sequence.</param>
/// <param name="num">Number of contiguous sequence slots to reserve.</param>
public void SkipMsgs(ulong seq, ulong num) public void SkipMsgs(ulong seq, ulong num)
{ {
if (seq != 0) if (seq != 0)
@@ -1973,6 +2079,8 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
/// future memory-pressure eviction removes entries from <c>_meta</c>. /// future memory-pressure eviction removes entries from <c>_meta</c>.
/// Reference: golang/nats-server/server/filestore.go:8308 (LoadMsg). /// Reference: golang/nats-server/server/filestore.go:8308 (LoadMsg).
/// </summary> /// </summary>
/// <param name="seq">Exact sequence to load.</param>
/// <param name="sm">Optional reusable output container.</param>
public StoreMsg LoadMsg(ulong seq, StoreMsg? sm) public StoreMsg LoadMsg(ulong seq, StoreMsg? sm)
{ {
var stored = MaterializeMessage(seq); var stored = MaterializeMessage(seq);
@@ -2044,6 +2152,8 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
/// Throws <see cref="KeyNotFoundException"/> if no message exists on the subject. /// Throws <see cref="KeyNotFoundException"/> if no message exists on the subject.
/// Reference: golang/nats-server/server/filestore.go — LoadLastMsg. /// Reference: golang/nats-server/server/filestore.go — LoadLastMsg.
/// </summary> /// </summary>
/// <param name="subject">Subject filter for selecting the latest message.</param>
/// <param name="sm">Optional reusable output container.</param>
public StoreMsg LoadLastMsg(string subject, StoreMsg? sm) public StoreMsg LoadLastMsg(string subject, StoreMsg? sm)
{ {
ulong? bestSeq = null; ulong? bestSeq = null;
@@ -2077,6 +2187,10 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
/// sequences skipped to reach it. /// sequences skipped to reach it.
/// Reference: golang/nats-server/server/filestore.go — LoadNextMsg. /// Reference: golang/nats-server/server/filestore.go — LoadNextMsg.
/// </summary> /// </summary>
/// <param name="filter">Subject filter with wildcard support.</param>
/// <param name="wc">Indicates whether caller precomputed filter as wildcard.</param>
/// <param name="start">Inclusive start sequence.</param>
/// <param name="sm">Optional reusable output container.</param>
public (StoreMsg Msg, ulong Skip) LoadNextMsg(string filter, bool wc, ulong start, StoreMsg? sm) public (StoreMsg Msg, ulong Skip) LoadNextMsg(string filter, bool wc, ulong start, StoreMsg? sm)
{ {
ulong? bestSeq = null; ulong? bestSeq = null;
@@ -2133,6 +2247,9 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
/// <paramref name="maxAllowed"/> results. /// <paramref name="maxAllowed"/> results.
/// Reference: golang/nats-server/server/filestore.go — MultiLastSeqs. /// Reference: golang/nats-server/server/filestore.go — MultiLastSeqs.
/// </summary> /// </summary>
/// <param name="filters">Subject filters used to select candidate subjects.</param>
/// <param name="maxSeq">Maximum allowed sequence in result set (0 means no cap).</param>
/// <param name="maxAllowed">Maximum number of results allowed before throwing.</param>
public ulong[] MultiLastSeqs(string[] filters, ulong maxSeq, int maxAllowed) public ulong[] MultiLastSeqs(string[] filters, ulong maxSeq, int maxAllowed)
{ {
var lastPerSubject = new Dictionary<string, ulong>(StringComparer.Ordinal); var lastPerSubject = new Dictionary<string, ulong>(StringComparer.Ordinal);
@@ -2166,6 +2283,7 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
/// Throws <see cref="KeyNotFoundException"/> if the sequence does not exist. /// Throws <see cref="KeyNotFoundException"/> if the sequence does not exist.
/// Reference: golang/nats-server/server/filestore.go — SubjectForSeq. /// Reference: golang/nats-server/server/filestore.go — SubjectForSeq.
/// </summary> /// </summary>
/// <param name="seq">Sequence to inspect.</param>
public string SubjectForSeq(ulong seq) public string SubjectForSeq(ulong seq)
{ {
if (!_meta.TryGetValue(seq, out var meta)) if (!_meta.TryGetValue(seq, out var meta))
@@ -2180,6 +2298,9 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
/// Returns (total, validThrough) where validThrough is the last sequence checked. /// Returns (total, validThrough) where validThrough is the last sequence checked.
/// Reference: golang/nats-server/server/filestore.go — NumPending. /// Reference: golang/nats-server/server/filestore.go — NumPending.
/// </summary> /// </summary>
/// <param name="sseq">Starting sequence lower bound.</param>
/// <param name="filter">Optional subject filter.</param>
/// <param name="lastPerSubject">Whether to count only one newest message per subject.</param>
public (ulong Total, ulong ValidThrough) NumPending(ulong sseq, string filter, bool lastPerSubject) public (ulong Total, ulong ValidThrough) NumPending(ulong sseq, string filter, bool lastPerSubject)
{ {
var candidates = _meta var candidates = _meta
@@ -2220,6 +2341,13 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
/// sequence.</para> /// sequence.</para>
/// Reference: golang/nats-server/server/filestore.go:6756 (storeRawMsg). /// Reference: golang/nats-server/server/filestore.go:6756 (storeRawMsg).
/// </summary> /// </summary>
/// <param name="subject">Publish subject for the replicated message.</param>
/// <param name="hdr">Optional protocol headers.</param>
/// <param name="msg">Message payload bytes.</param>
/// <param name="seq">Sequence assigned by replication source.</param>
/// <param name="ts">Nanosecond timestamp assigned by replication source.</param>
/// <param name="ttl">Optional per-message TTL in nanoseconds.</param>
/// <param name="discardNewCheck">Reserved parity flag for discard-new behavior.</param>
public void StoreRawMsg(string subject, byte[]? hdr, byte[] msg, ulong seq, long ts, long ttl, bool discardNewCheck) public void StoreRawMsg(string subject, byte[]? hdr, byte[] msg, ulong seq, long ts, long ttl, bool discardNewCheck)
{ {
if (_stopped) if (_stopped)
@@ -2263,6 +2391,8 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
/// Throws <see cref="KeyNotFoundException"/> if no such message exists. /// Throws <see cref="KeyNotFoundException"/> if no such message exists.
/// Reference: golang/nats-server/server/filestore.go — LoadPrevMsg. /// Reference: golang/nats-server/server/filestore.go — LoadPrevMsg.
/// </summary> /// </summary>
/// <param name="start">Exclusive upper sequence bound.</param>
/// <param name="sm">Optional reusable output container.</param>
public StoreMsg LoadPrevMsg(ulong start, StoreMsg? sm) public StoreMsg LoadPrevMsg(ulong start, StoreMsg? sm)
{ {
if (start == 0) if (start == 0)
@@ -2334,6 +2464,7 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
/// encoding will be added when the RAFT snapshot codec is implemented (Task 9). /// encoding will be added when the RAFT snapshot codec is implemented (Task 9).
/// Reference: golang/nats-server/server/filestore.go — EncodedStreamState. /// Reference: golang/nats-server/server/filestore.go — EncodedStreamState.
/// </summary> /// </summary>
/// <param name="failed">Number of failed apply operations from consensus layer.</param>
public byte[] EncodedStreamState(ulong failed) => []; public byte[] EncodedStreamState(ulong failed) => [];
/// <summary> /// <summary>
@@ -2341,6 +2472,7 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
/// MaxAge, etc.) to the store options. /// MaxAge, etc.) to the store options.
/// Reference: golang/nats-server/server/filestore.go — UpdateConfig. /// Reference: golang/nats-server/server/filestore.go — UpdateConfig.
/// </summary> /// </summary>
/// <param name="cfg">Updated stream configuration.</param>
public void UpdateConfig(StreamConfig cfg) public void UpdateConfig(StreamConfig cfg)
{ {
_options.MaxMsgsPerSubject = cfg.MaxMsgsPer; _options.MaxMsgsPerSubject = cfg.MaxMsgsPer;
@@ -2405,6 +2537,9 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
/// matching the Go server's consumer directory layout. /// matching the Go server's consumer directory layout.
/// Reference: golang/nats-server/server/filestore.go — newConsumerFileStore. /// Reference: golang/nats-server/server/filestore.go — newConsumerFileStore.
/// </summary> /// </summary>
/// <param name="name">Consumer durable name.</param>
/// <param name="created">Consumer creation timestamp.</param>
/// <param name="cfg">Consumer configuration snapshot.</param>
public IConsumerStore ConsumerStore(string name, DateTime created, ConsumerConfig cfg) public IConsumerStore ConsumerStore(string name, DateTime created, ConsumerConfig cfg)
{ {
var consumerDir = Path.Combine(_options.Directory, "obs", name); var consumerDir = Path.Combine(_options.Directory, "obs", name);
@@ -2428,6 +2563,7 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
/// </summary> /// </summary>
public async Task FlushAllPending() public async Task FlushAllPending()
{ {
DrainSyncQueue();
_activeBlock?.FlushPending(); _activeBlock?.FlushPending();
_activeBlock?.Flush(); _activeBlock?.Flush();
await WriteStreamStateAsync(); await WriteStreamStateAsync();
@@ -2445,7 +2581,7 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
while (!ct.IsCancellationRequested) while (!ct.IsCancellationRequested)
{ {
try { await _flushSignal.Reader.WaitToReadAsync(ct); } try { await _flushSignal.Reader.WaitToReadAsync(ct); }
catch (OperationCanceledException) { return; } catch (OperationCanceledException) { break; }
_flushSignal.Reader.TryRead(out _); _flushSignal.Reader.TryRead(out _);
var block = _activeBlock; var block = _activeBlock;
@@ -2465,6 +2601,26 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
} }
block.FlushPending(); block.FlushPending();
// Drain deferred sync queue — fsync sealed blocks outside the hot path.
// Go: filestore.go:7242-7266 — sync happens after releasing the block lock.
DrainSyncQueue();
}
// Final drain on shutdown to ensure all data reaches stable storage.
DrainSyncQueue();
}
/// <summary>
/// Fsyncs all blocks queued by <see cref="RotateBlock"/> for deferred sync.
/// Called from the background flush loop and on shutdown.
/// </summary>
private void DrainSyncQueue()
{
while (_needSyncBlocks.TryDequeue(out var syncBlock))
{
try { syncBlock.Flush(); }
catch (ObjectDisposedException) { /* Block was already disposed (e.g. purge). */ }
} }
} }
@@ -2528,18 +2684,45 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
private sealed record StreamStateSnapshot private sealed record StreamStateSnapshot
{ {
/// <summary>
/// First live sequence at checkpoint time.
/// </summary>
public ulong FirstSeq { get; init; } public ulong FirstSeq { get; init; }
/// <summary>
/// Last seen sequence watermark at checkpoint time.
/// </summary>
public ulong LastSeq { get; init; } public ulong LastSeq { get; init; }
/// <summary>
/// Number of live messages at checkpoint time.
/// </summary>
public ulong Messages { get; init; } public ulong Messages { get; init; }
/// <summary>
/// Approximate bytes written across active blocks at checkpoint time.
/// </summary>
public ulong Bytes { get; init; } public ulong Bytes { get; init; }
} }
private sealed class FileRecord private sealed class FileRecord
{ {
/// <summary>
/// Stream sequence for this snapshot record.
/// </summary>
public ulong Sequence { get; init; } public ulong Sequence { get; init; }
/// <summary>
/// Subject associated with the message.
/// </summary>
public string? Subject { get; init; } public string? Subject { get; init; }
/// <summary>
/// Optional base64-encoded protocol headers.
/// </summary>
public string? HeadersBase64 { get; init; } public string? HeadersBase64 { get; init; }
/// <summary>
/// Base64-encoded persisted payload.
/// </summary>
public string? PayloadBase64 { get; init; } public string? PayloadBase64 { get; init; }
/// <summary>
/// Original message timestamp in UTC.
/// </summary>
public DateTime TimestampUtc { get; init; } public DateTime TimestampUtc { get; init; }
} }
@@ -2570,8 +2753,17 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
/// <summary>Tracks per-block cache state.</summary> /// <summary>Tracks per-block cache state.</summary>
internal sealed class CacheEntry internal sealed class CacheEntry
{ {
/// <summary>
/// Block identifier for this cache entry.
/// </summary>
public int BlockId { get; init; } public int BlockId { get; init; }
/// <summary>
/// Last write time in Environment.TickCount64 milliseconds.
/// </summary>
public long LastWriteTime { get; set; } // Environment.TickCount64 (ms) public long LastWriteTime { get; set; } // Environment.TickCount64 (ms)
/// <summary>
/// Approximate bytes currently buffered for this block.
/// </summary>
public long ApproximateBytes { get; set; } public long ApproximateBytes { get; set; }
} }
@@ -2623,6 +2815,8 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
/// Records a write to the specified block, updating the entry's timestamp and size. /// Records a write to the specified block, updating the entry's timestamp and size.
/// Reference: golang/nats-server/server/filestore.go:6529 (lwts update on write). /// Reference: golang/nats-server/server/filestore.go:6529 (lwts update on write).
/// </summary> /// </summary>
/// <param name="blockId">Block id receiving the write.</param>
/// <param name="bytes">Approximate bytes added to cache.</param>
public void TrackWrite(int blockId, long bytes) public void TrackWrite(int blockId, long bytes)
{ {
var now = Environment.TickCount64; var now = Environment.TickCount64;
@@ -2642,6 +2836,9 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
/// Allows tests to simulate past writes without sleeping, avoiding timing /// Allows tests to simulate past writes without sleeping, avoiding timing
/// dependencies in TTL and size-cap eviction tests. /// dependencies in TTL and size-cap eviction tests.
/// </summary> /// </summary>
/// <param name="blockId">Block id receiving the synthetic write.</param>
/// <param name="bytes">Approximate bytes added to cache.</param>
/// <param name="tickCount64Ms">Synthetic write timestamp in TickCount64 milliseconds.</param>
internal void TrackWriteAt(int blockId, long bytes, long tickCount64Ms) internal void TrackWriteAt(int blockId, long bytes, long tickCount64Ms)
{ {
_entries.AddOrUpdate( _entries.AddOrUpdate(
@@ -2660,6 +2857,7 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
/// Called from <see cref="FileStore.RotateBlock"/> for the outgoing block. /// Called from <see cref="FileStore.RotateBlock"/> for the outgoing block.
/// Reference: golang/nats-server/server/filestore.go:4499 (flushPendingMsgsLocked on rotation). /// Reference: golang/nats-server/server/filestore.go:4499 (flushPendingMsgsLocked on rotation).
/// </summary> /// </summary>
/// <param name="blockId">Block id to evict.</param>
public void EvictBlock(int blockId) public void EvictBlock(int blockId)
{ {
if (!_entries.TryRemove(blockId, out _)) if (!_entries.TryRemove(blockId, out _))
@@ -2673,6 +2871,17 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
block.ClearCache(); block.ClearCache();
} }
/// <summary>
/// Removes the cache entry for the specified block without calling Flush (no fsync).
/// The caller is responsible for ensuring the block is synced later (e.g. via the
/// background flush loop's deferred sync queue).
/// Used by <see cref="FileStore.RotateBlock"/> to avoid synchronous fsync on the hot path.
/// </summary>
public void EvictBlockNoSync(int blockId)
{
_entries.TryRemove(blockId, out _);
}
/// <summary> /// <summary>
/// Flushes and clears the cache for all currently tracked blocks. /// Flushes and clears the cache for all currently tracked blocks.
/// Reference: golang/nats-server/server/filestore.go:5499 (flushPendingMsgsLocked, all blocks). /// Reference: golang/nats-server/server/filestore.go:5499 (flushPendingMsgsLocked, all blocks).
@@ -4,43 +4,58 @@ namespace NATS.Server.JetStream.Storage;
public sealed class FileStoreOptions public sealed class FileStoreOptions
{ {
/// <summary>Root directory where JetStream file store data is persisted.</summary>
public string Directory { get; set; } = string.Empty; public string Directory { get; set; } = string.Empty;
/// <summary>Block size used for stream data files in bytes.</summary>
public int BlockSizeBytes { get; set; } = 64 * 1024; public int BlockSizeBytes { get; set; } = 64 * 1024;
/// <summary>Manifest file name that tracks index metadata for stream blocks.</summary>
public string IndexManifestFileName { get; set; } = "index.manifest.json"; public string IndexManifestFileName { get; set; } = "index.manifest.json";
/// <summary>Maximum message age in milliseconds before retention eviction.</summary>
public int MaxAgeMs { get; set; } public int MaxAgeMs { get; set; }
// Go: StreamConfig.MaxBytes — maximum total bytes for the stream. // Go: StreamConfig.MaxBytes — maximum total bytes for the stream.
// Reference: golang/nats-server/server/filestore.go — maxBytes field. // Reference: golang/nats-server/server/filestore.go — maxBytes field.
/// <summary>Maximum total bytes retained by the stream across all subjects.</summary>
public long MaxBytes { get; set; } public long MaxBytes { get; set; }
// Go: StreamConfig.Discard — discard policy (Old or New). // Go: StreamConfig.Discard — discard policy (Old or New).
// Reference: golang/nats-server/server/filestore.go — discardPolicy field. // Reference: golang/nats-server/server/filestore.go — discardPolicy field.
/// <summary>Discard strategy applied when limits are reached.</summary>
public DiscardPolicy Discard { get; set; } = DiscardPolicy.Old; public DiscardPolicy Discard { get; set; } = DiscardPolicy.Old;
// Legacy boolean compression / encryption flags (FSV1 envelope format). // Legacy boolean compression / encryption flags (FSV1 envelope format).
// When set and the corresponding enum is left at its default (NoCompression / // When set and the corresponding enum is left at its default (NoCompression /
// NoCipher), the legacy Deflate / XOR path is used for backward compatibility. // NoCipher), the legacy Deflate / XOR path is used for backward compatibility.
/// <summary>Enables legacy compression behavior for backward-compatible file envelopes.</summary>
public bool EnableCompression { get; set; } public bool EnableCompression { get; set; }
/// <summary>Enables legacy encryption behavior for backward-compatible file envelopes.</summary>
public bool EnableEncryption { get; set; } public bool EnableEncryption { get; set; }
/// <summary>When enabled, verifies payload checksums during load and replay.</summary>
public bool EnablePayloadIntegrityChecks { get; set; } = true; public bool EnablePayloadIntegrityChecks { get; set; } = true;
/// <summary>Raw key material used by encrypted file store modes.</summary>
public byte[]? EncryptionKey { get; set; } public byte[]? EncryptionKey { get; set; }
// Go parity: StoreCompression / StoreCipher (filestore.go ~line 91-92). // Go parity: StoreCompression / StoreCipher (filestore.go ~line 91-92).
// When Compression == S2Compression the S2/Snappy codec is used (FSV2 envelope). // When Compression == S2Compression the S2/Snappy codec is used (FSV2 envelope).
// When Cipher != NoCipher an AEAD cipher is used instead of the legacy XOR. // When Cipher != NoCipher an AEAD cipher is used instead of the legacy XOR.
// Enums are defined in AeadEncryptor.cs. // Enums are defined in AeadEncryptor.cs.
/// <summary>Compression algorithm used for new file store blocks.</summary>
public StoreCompression Compression { get; set; } = StoreCompression.NoCompression; public StoreCompression Compression { get; set; } = StoreCompression.NoCompression;
/// <summary>Cipher suite used for new file store blocks.</summary>
public StoreCipher Cipher { get; set; } = StoreCipher.NoCipher; public StoreCipher Cipher { get; set; } = StoreCipher.NoCipher;
// Go: StreamConfig.MaxMsgsPer — maximum messages per subject (1 = keep last per subject). // Go: StreamConfig.MaxMsgsPer — maximum messages per subject (1 = keep last per subject).
// Reference: golang/nats-server/server/filestore.go — per-subject message limits. // Reference: golang/nats-server/server/filestore.go — per-subject message limits.
/// <summary>Maximum retained message count per subject.</summary>
public int MaxMsgsPerSubject { get; set; } public int MaxMsgsPerSubject { get; set; }
// Go: filestore.go:4443 (setupWriteCache) — bounded write-cache settings. // Go: filestore.go:4443 (setupWriteCache) — bounded write-cache settings.
// MaxCacheSize: total bytes across all cached blocks before eviction kicks in. // MaxCacheSize: total bytes across all cached blocks before eviction kicks in.
// CacheExpiry: TTL after which an idle block's cache is flushed and cleared. // CacheExpiry: TTL after which an idle block's cache is flushed and cleared.
// Reference: golang/nats-server/server/filestore.go:6220 (expireCacheLocked). // Reference: golang/nats-server/server/filestore.go:6220 (expireCacheLocked).
/// <summary>Upper bound for in-memory block cache usage before eviction.</summary>
public long MaxCacheSize { get; set; } = 64 * 1024 * 1024; // 64 MB default public long MaxCacheSize { get; set; } = 64 * 1024 * 1024; // 64 MB default
/// <summary>Idle time after which cached block data is expired.</summary>
public TimeSpan CacheExpiry { get; set; } = TimeSpan.FromSeconds(2); public TimeSpan CacheExpiry { get; set; } = TimeSpan.FromSeconds(2);
} }
@@ -12,45 +12,69 @@ namespace NATS.Server.JetStream.Storage;
public interface IConsumerStore public interface IConsumerStore
{ {
// Go: ConsumerStore.SetStarting — initialise the starting stream sequence for a new consumer // Go: ConsumerStore.SetStarting — initialise the starting stream sequence for a new consumer
/// <summary>Sets the initial stream sequence from which consumer delivery begins.</summary>
/// <param name="sseq">Starting stream sequence for this consumer.</param>
void SetStarting(ulong sseq); void SetStarting(ulong sseq);
// Go: ConsumerStore.UpdateStarting — update the starting sequence after a reset // Go: ConsumerStore.UpdateStarting — update the starting sequence after a reset
/// <summary>Updates the persisted start sequence after consumer reconfiguration or replay reset.</summary>
/// <param name="sseq">New starting stream sequence.</param>
void UpdateStarting(ulong sseq); void UpdateStarting(ulong sseq);
// Go: ConsumerStore.Reset — reset state to a given stream sequence // Go: ConsumerStore.Reset — reset state to a given stream sequence
/// <summary>Resets consumer progress and pending state to a specific stream sequence.</summary>
/// <param name="sseq">Stream sequence used as the reset baseline.</param>
void Reset(ulong sseq); void Reset(ulong sseq);
// Go: ConsumerStore.HasState — returns true if any persisted state exists // Go: ConsumerStore.HasState — returns true if any persisted state exists
/// <summary>Indicates whether durable state for the consumer exists in storage.</summary>
bool HasState(); bool HasState();
// Go: ConsumerStore.UpdateDelivered — record a new delivery (dseq=consumer seq, sseq=stream seq, // Go: ConsumerStore.UpdateDelivered — record a new delivery (dseq=consumer seq, sseq=stream seq,
// dc=delivery count, ts=Unix nanosecond timestamp) // dc=delivery count, ts=Unix nanosecond timestamp)
/// <summary>Records an attempted delivery so replay and redelivery bookkeeping remain durable.</summary>
/// <param name="dseq">Consumer delivery sequence number.</param>
/// <param name="sseq">Source stream sequence delivered to the consumer.</param>
/// <param name="dc">Delivery attempt count for this stream message.</param>
/// <param name="ts">Delivery timestamp in Unix nanoseconds.</param>
void UpdateDelivered(ulong dseq, ulong sseq, ulong dc, long ts); void UpdateDelivered(ulong dseq, ulong sseq, ulong dc, long ts);
// Go: ConsumerStore.UpdateAcks — record an acknowledgement (dseq=consumer seq, sseq=stream seq) // Go: ConsumerStore.UpdateAcks — record an acknowledgement (dseq=consumer seq, sseq=stream seq)
/// <summary>Persists acknowledgement progress so ack floors survive restart and failover.</summary>
/// <param name="dseq">Acknowledged consumer delivery sequence.</param>
/// <param name="sseq">Acknowledged source stream sequence.</param>
void UpdateAcks(ulong dseq, ulong sseq); void UpdateAcks(ulong dseq, ulong sseq);
// Go: ConsumerStore.Update — overwrite the full consumer state in one call // Go: ConsumerStore.Update — overwrite the full consumer state in one call
/// <summary>Overwrites the full persisted consumer state snapshot.</summary>
/// <param name="state">Complete consumer state to persist.</param>
void Update(ConsumerState state); void Update(ConsumerState state);
// Go: ConsumerStore.State — return a snapshot of current consumer state // Go: ConsumerStore.State — return a snapshot of current consumer state
/// <summary>Returns a copy of current persisted consumer state.</summary>
ConsumerState State(); ConsumerState State();
// Go: ConsumerStore.BorrowState — return state without copying (caller must not retain beyond call) // Go: ConsumerStore.BorrowState — return state without copying (caller must not retain beyond call)
/// <summary>Returns a non-copied state view for short-lived internal access.</summary>
ConsumerState BorrowState(); ConsumerState BorrowState();
// Go: ConsumerStore.EncodedState — return the binary-encoded state for replication // Go: ConsumerStore.EncodedState — return the binary-encoded state for replication
/// <summary>Returns binary-encoded consumer state for replication and snapshot transfer.</summary>
byte[] EncodedState(); byte[] EncodedState();
// Go: ConsumerStore.Type — the storage type backing this store (File or Memory) // Go: ConsumerStore.Type — the storage type backing this store (File or Memory)
/// <summary>Returns the backing storage type used by this consumer store.</summary>
StorageType Type(); StorageType Type();
// Go: ConsumerStore.Stop — flush and close the store without deleting data // Go: ConsumerStore.Stop — flush and close the store without deleting data
/// <summary>Flushes and closes the store while retaining persisted consumer state.</summary>
void Stop(); void Stop();
// Go: ConsumerStore.Delete — stop the store and delete all persisted state // Go: ConsumerStore.Delete — stop the store and delete all persisted state
/// <summary>Deletes all persisted consumer state and releases underlying resources.</summary>
void Delete(); void Delete();
// Go: ConsumerStore.StreamDelete — called when the parent stream is deleted // Go: ConsumerStore.StreamDelete — called when the parent stream is deleted
/// <summary>Handles parent stream deletion and cleans consumer persistence accordingly.</summary>
void StreamDelete(); void StreamDelete();
} }
@@ -21,9 +21,13 @@ public sealed class MemStore : IStreamStore
private sealed class SnapshotRecord private sealed class SnapshotRecord
{ {
/// <summary>Stream sequence for the captured message.</summary>
public ulong Sequence { get; init; } public ulong Sequence { get; init; }
/// <summary>Published subject for the captured message.</summary>
public string Subject { get; init; } = string.Empty; public string Subject { get; init; } = string.Empty;
/// <summary>Base64-encoded payload bytes persisted in the snapshot.</summary>
public string PayloadBase64 { get; init; } = string.Empty; public string PayloadBase64 { get; init; } = string.Empty;
/// <summary>Original message timestamp in UTC.</summary>
public DateTime TimestampUtc { get; init; } public DateTime TimestampUtc { get; init; }
} }
@@ -39,6 +43,14 @@ public sealed class MemStore : IStreamStore
public readonly ulong Seq; public readonly ulong Seq;
public readonly long Ts; // Unix nanoseconds public readonly long Ts; // Unix nanoseconds
/// <summary>
/// Creates the in-memory representation for a single stream message.
/// </summary>
/// <param name="subj">Subject the message was published to.</param>
/// <param name="hdr">Optional NATS header bytes.</param>
/// <param name="data">Optional payload bytes.</param>
/// <param name="seq">Assigned stream sequence.</param>
/// <param name="ts">Publish timestamp in Unix nanoseconds.</param>
public Msg(string subj, byte[]? hdr, byte[]? data, ulong seq, long ts) public Msg(string subj, byte[]? hdr, byte[]? data, ulong seq, long ts)
{ {
Subj = subj; Subj = subj;
@@ -106,8 +118,15 @@ public sealed class MemStore : IStreamStore
// Constructor // Constructor
// ------------------------------------------------------------------------- // -------------------------------------------------------------------------
/// <summary>
/// Initializes an empty in-memory stream store with default limits.
/// </summary>
public MemStore() { } public MemStore() { }
/// <summary>
/// Initializes an in-memory stream store using stream retention and TTL configuration.
/// </summary>
/// <param name="cfg">Stream configuration used to seed limits and sequence watermark state.</param>
public MemStore(StreamConfig cfg) public MemStore(StreamConfig cfg)
{ {
_cfg = cfg; _cfg = cfg;
@@ -123,9 +142,13 @@ public sealed class MemStore : IStreamStore
} }
// IStreamStore cached state properties — O(1), maintained incrementally. // IStreamStore cached state properties — O(1), maintained incrementally.
/// <summary>Gets the highest sequence assigned by this stream store.</summary>
public ulong LastSeq { get { lock (_gate) return _st.LastSeq; } } public ulong LastSeq { get { lock (_gate) return _st.LastSeq; } }
/// <summary>Gets the current number of retained messages.</summary>
public ulong MessageCount { get { lock (_gate) return _st.Msgs; } } public ulong MessageCount { get { lock (_gate) return _st.Msgs; } }
/// <summary>Gets the total byte usage for retained messages.</summary>
public ulong TotalBytes { get { lock (_gate) return _st.Bytes; } } public ulong TotalBytes { get { lock (_gate) return _st.Bytes; } }
/// <inheritdoc />
ulong IStreamStore.FirstSeq { get { lock (_gate) return _st.Msgs == 0 ? (_st.FirstSeq > 0 ? _st.FirstSeq : 0UL) : _st.FirstSeq; } } ulong IStreamStore.FirstSeq { get { lock (_gate) return _st.Msgs == 0 ? (_st.FirstSeq > 0 ? _st.FirstSeq : 0UL) : _st.FirstSeq; } }
// ------------------------------------------------------------------------- // -------------------------------------------------------------------------
@@ -133,6 +156,13 @@ public sealed class MemStore : IStreamStore
// ------------------------------------------------------------------------- // -------------------------------------------------------------------------
// Go: memStore.StoreMsg — async wrapper // Go: memStore.StoreMsg — async wrapper
/// <summary>
/// Appends a new message to the stream and returns the assigned sequence.
/// </summary>
/// <param name="subject">Subject used for stream indexing and filtering.</param>
/// <param name="payload">Message payload bytes.</param>
/// <param name="ct">Cancellation token reserved for API parity.</param>
/// <returns>The sequence assigned to the stored message.</returns>
public ValueTask<ulong> AppendAsync(string subject, ReadOnlyMemory<byte> payload, CancellationToken ct) public ValueTask<ulong> AppendAsync(string subject, ReadOnlyMemory<byte> payload, CancellationToken ct)
{ {
lock (_gate) lock (_gate)
@@ -144,6 +174,12 @@ public sealed class MemStore : IStreamStore
} }
} }
/// <summary>
/// Loads a stored message by its stream sequence.
/// </summary>
/// <param name="sequence">Sequence to load.</param>
/// <param name="ct">Cancellation token reserved for API parity.</param>
/// <returns>The stored message, or <see langword="null"/> when the sequence is absent.</returns>
public ValueTask<StoredMessage?> LoadAsync(ulong sequence, CancellationToken ct) public ValueTask<StoredMessage?> LoadAsync(ulong sequence, CancellationToken ct)
{ {
lock (_gate) lock (_gate)
@@ -160,6 +196,12 @@ public sealed class MemStore : IStreamStore
} }
} }
/// <summary>
/// Loads the most recent retained message for a concrete subject.
/// </summary>
/// <param name="subject">Subject to query.</param>
/// <param name="ct">Cancellation token reserved for API parity.</param>
/// <returns>The latest message for the subject, or <see langword="null"/> when none exists.</returns>
public ValueTask<StoredMessage?> LoadLastBySubjectAsync(string subject, CancellationToken ct) public ValueTask<StoredMessage?> LoadLastBySubjectAsync(string subject, CancellationToken ct)
{ {
lock (_gate) lock (_gate)
@@ -178,6 +220,11 @@ public sealed class MemStore : IStreamStore
} }
} }
/// <summary>
/// Lists all retained messages in ascending sequence order.
/// </summary>
/// <param name="ct">Cancellation token reserved for API parity.</param>
/// <returns>Snapshot of retained messages.</returns>
public ValueTask<IReadOnlyList<StoredMessage>> ListAsync(CancellationToken ct) public ValueTask<IReadOnlyList<StoredMessage>> ListAsync(CancellationToken ct)
{ {
lock (_gate) lock (_gate)
@@ -196,6 +243,12 @@ public sealed class MemStore : IStreamStore
} }
} }
/// <summary>
/// Soft-deletes a message by sequence.
/// </summary>
/// <param name="sequence">Sequence to remove.</param>
/// <param name="ct">Cancellation token reserved for API parity.</param>
/// <returns><see langword="true"/> when the message existed and was removed.</returns>
public ValueTask<bool> RemoveAsync(ulong sequence, CancellationToken ct) public ValueTask<bool> RemoveAsync(ulong sequence, CancellationToken ct)
{ {
lock (_gate) lock (_gate)
@@ -204,6 +257,10 @@ public sealed class MemStore : IStreamStore
} }
} }
/// <summary>
/// Removes all retained messages while preserving next-sequence continuity.
/// </summary>
/// <param name="ct">Cancellation token reserved for API parity.</param>
public ValueTask PurgeAsync(CancellationToken ct) public ValueTask PurgeAsync(CancellationToken ct)
{ {
lock (_gate) lock (_gate)
@@ -213,6 +270,11 @@ public sealed class MemStore : IStreamStore
} }
} }
/// <summary>
/// Creates a JSON snapshot of retained messages for backup and restore workflows.
/// </summary>
/// <param name="ct">Cancellation token reserved for API parity.</param>
/// <returns>UTF-8 JSON payload representing stream messages.</returns>
public ValueTask<byte[]> CreateSnapshotAsync(CancellationToken ct) public ValueTask<byte[]> CreateSnapshotAsync(CancellationToken ct)
{ {
lock (_gate) lock (_gate)
@@ -232,6 +294,11 @@ public sealed class MemStore : IStreamStore
} }
} }
/// <summary>
/// Restores the stream from a previously captured snapshot payload.
/// </summary>
/// <param name="snapshot">Serialized snapshot bytes.</param>
/// <param name="ct">Cancellation token reserved for API parity.</param>
public ValueTask RestoreSnapshotAsync(ReadOnlyMemory<byte> snapshot, CancellationToken ct) public ValueTask RestoreSnapshotAsync(ReadOnlyMemory<byte> snapshot, CancellationToken ct)
{ {
lock (_gate) lock (_gate)
@@ -266,6 +333,11 @@ public sealed class MemStore : IStreamStore
} }
} }
/// <summary>
/// Returns API state counters used by JetStream management endpoints.
/// </summary>
/// <param name="ct">Cancellation token reserved for API parity.</param>
/// <returns>Current message, sequence, and byte counters.</returns>
public ValueTask<ApiStreamState> GetStateAsync(CancellationToken ct) public ValueTask<ApiStreamState> GetStateAsync(CancellationToken ct)
{ {
lock (_gate) lock (_gate)
@@ -290,6 +362,7 @@ public sealed class MemStore : IStreamStore
// ------------------------------------------------------------------------- // -------------------------------------------------------------------------
// Go: memStore.StoreMsg server/memstore.go:350 // Go: memStore.StoreMsg server/memstore.go:350
/// <inheritdoc />
(ulong Seq, long Ts) IStreamStore.StoreMsg(string subject, byte[]? hdr, byte[] msg, long ttl) (ulong Seq, long Ts) IStreamStore.StoreMsg(string subject, byte[]? hdr, byte[] msg, long ttl)
{ {
lock (_gate) lock (_gate)
@@ -303,6 +376,7 @@ public sealed class MemStore : IStreamStore
} }
// Go: memStore.StoreRawMsg server/memstore.go:329 // Go: memStore.StoreRawMsg server/memstore.go:329
/// <inheritdoc />
void IStreamStore.StoreRawMsg(string subject, byte[]? hdr, byte[] msg, ulong seq, long ts, long ttl, bool discardNewCheck) void IStreamStore.StoreRawMsg(string subject, byte[]? hdr, byte[] msg, ulong seq, long ts, long ttl, bool discardNewCheck)
{ {
lock (_gate) lock (_gate)
@@ -312,6 +386,7 @@ public sealed class MemStore : IStreamStore
} }
// Go: memStore.SkipMsg server/memstore.go:368 // Go: memStore.SkipMsg server/memstore.go:368
/// <inheritdoc />
ulong IStreamStore.SkipMsg(ulong seq) ulong IStreamStore.SkipMsg(ulong seq)
{ {
lock (_gate) lock (_gate)
@@ -336,6 +411,7 @@ public sealed class MemStore : IStreamStore
} }
// Go: memStore.SkipMsgs server/memstore.go:395 // Go: memStore.SkipMsgs server/memstore.go:395
/// <inheritdoc />
void IStreamStore.SkipMsgs(ulong seq, ulong num) void IStreamStore.SkipMsgs(ulong seq, ulong num)
{ {
lock (_gate) lock (_gate)
@@ -361,9 +437,11 @@ public sealed class MemStore : IStreamStore
} }
// Go: memStore.FlushAllPending server/memstore.go:423 — no-op for in-memory store // Go: memStore.FlushAllPending server/memstore.go:423 — no-op for in-memory store
/// <inheritdoc />
Task IStreamStore.FlushAllPending() => Task.CompletedTask; Task IStreamStore.FlushAllPending() => Task.CompletedTask;
// Go: memStore.LoadMsg server/memstore.go:1692 // Go: memStore.LoadMsg server/memstore.go:1692
/// <inheritdoc />
StoreMsg IStreamStore.LoadMsg(ulong seq, StoreMsg? sm) StoreMsg IStreamStore.LoadMsg(ulong seq, StoreMsg? sm)
{ {
lock (_gate) lock (_gate)
@@ -375,6 +453,7 @@ public sealed class MemStore : IStreamStore
} }
// Go: memStore.LoadNextMsg server/memstore.go:1798 // Go: memStore.LoadNextMsg server/memstore.go:1798
/// <inheritdoc />
(StoreMsg Msg, ulong Skip) IStreamStore.LoadNextMsg(string filter, bool wc, ulong start, StoreMsg? sm) (StoreMsg Msg, ulong Skip) IStreamStore.LoadNextMsg(string filter, bool wc, ulong start, StoreMsg? sm)
{ {
lock (_gate) lock (_gate)
@@ -397,6 +476,7 @@ public sealed class MemStore : IStreamStore
} }
// Go: memStore.LoadLastMsg server/memstore.go:1724 // Go: memStore.LoadLastMsg server/memstore.go:1724
/// <inheritdoc />
StoreMsg IStreamStore.LoadLastMsg(string subject, StoreMsg? sm) StoreMsg IStreamStore.LoadLastMsg(string subject, StoreMsg? sm)
{ {
lock (_gate) lock (_gate)
@@ -442,6 +522,7 @@ public sealed class MemStore : IStreamStore
} }
// Go: memStore.LoadPrevMsg — walk backwards from start // Go: memStore.LoadPrevMsg — walk backwards from start
/// <inheritdoc />
StoreMsg IStreamStore.LoadPrevMsg(ulong start, StoreMsg? sm) StoreMsg IStreamStore.LoadPrevMsg(ulong start, StoreMsg? sm)
{ {
lock (_gate) lock (_gate)
@@ -457,6 +538,7 @@ public sealed class MemStore : IStreamStore
} }
// Go: memStore.RemoveMsg — soft delete // Go: memStore.RemoveMsg — soft delete
/// <inheritdoc />
bool IStreamStore.RemoveMsg(ulong seq) bool IStreamStore.RemoveMsg(ulong seq)
{ {
lock (_gate) lock (_gate)
@@ -466,6 +548,7 @@ public sealed class MemStore : IStreamStore
} }
// Go: memStore.EraseMsg — overwrite then remove // Go: memStore.EraseMsg — overwrite then remove
/// <inheritdoc />
bool IStreamStore.EraseMsg(ulong seq) bool IStreamStore.EraseMsg(ulong seq)
{ {
lock (_gate) lock (_gate)
@@ -475,6 +558,7 @@ public sealed class MemStore : IStreamStore
} }
// Go: memStore.Purge server/memstore.go:1471 // Go: memStore.Purge server/memstore.go:1471
/// <inheritdoc />
ulong IStreamStore.Purge() ulong IStreamStore.Purge()
{ {
lock (_gate) lock (_gate)
@@ -484,6 +568,7 @@ public sealed class MemStore : IStreamStore
} }
// Go: memStore.PurgeEx server/memstore.go:1422 // Go: memStore.PurgeEx server/memstore.go:1422
/// <inheritdoc />
ulong IStreamStore.PurgeEx(string subject, ulong seq, ulong keep) ulong IStreamStore.PurgeEx(string subject, ulong seq, ulong keep)
{ {
if (string.IsNullOrEmpty(subject) || subject == ">") if (string.IsNullOrEmpty(subject) || subject == ">")
@@ -536,9 +621,11 @@ public sealed class MemStore : IStreamStore
} }
// Go: memStore.Compact server/memstore.go:1509 // Go: memStore.Compact server/memstore.go:1509
/// <inheritdoc />
ulong IStreamStore.Compact(ulong seq) => CompactInternal(seq); ulong IStreamStore.Compact(ulong seq) => CompactInternal(seq);
// Go: memStore.Truncate server/memstore.go:1618 // Go: memStore.Truncate server/memstore.go:1618
/// <inheritdoc />
void IStreamStore.Truncate(ulong seq) void IStreamStore.Truncate(ulong seq)
{ {
lock (_gate) lock (_gate)
@@ -574,6 +661,7 @@ public sealed class MemStore : IStreamStore
} }
// Go: memStore.GetSeqFromTime server/memstore.go:453 // Go: memStore.GetSeqFromTime server/memstore.go:453
/// <inheritdoc />
ulong IStreamStore.GetSeqFromTime(DateTime t) ulong IStreamStore.GetSeqFromTime(DateTime t)
{ {
lock (_gate) lock (_gate)
@@ -642,10 +730,12 @@ public sealed class MemStore : IStreamStore
} }
// Go: memStore.FilteredState server/memstore.go:531 // Go: memStore.FilteredState server/memstore.go:531
/// <inheritdoc />
SimpleState IStreamStore.FilteredState(ulong seq, string subject) SimpleState IStreamStore.FilteredState(ulong seq, string subject)
=> FilteredStateInternal(seq, subject); => FilteredStateInternal(seq, subject);
// Go: memStore.SubjectsState server/memstore.go:748 // Go: memStore.SubjectsState server/memstore.go:748
/// <inheritdoc />
Dictionary<string, SimpleState> IStreamStore.SubjectsState(string filterSubject) Dictionary<string, SimpleState> IStreamStore.SubjectsState(string filterSubject)
{ {
lock (_gate) lock (_gate)
@@ -668,6 +758,7 @@ public sealed class MemStore : IStreamStore
} }
// Go: memStore.SubjectsTotals server/memstore.go:881 // Go: memStore.SubjectsTotals server/memstore.go:881
/// <inheritdoc />
Dictionary<string, ulong> IStreamStore.SubjectsTotals(string filterSubject) Dictionary<string, ulong> IStreamStore.SubjectsTotals(string filterSubject)
{ {
lock (_gate) lock (_gate)
@@ -683,6 +774,7 @@ public sealed class MemStore : IStreamStore
} }
// Go: memStore.AllLastSeqs server/memstore.go:780 // Go: memStore.AllLastSeqs server/memstore.go:780
/// <inheritdoc />
ulong[] IStreamStore.AllLastSeqs() ulong[] IStreamStore.AllLastSeqs()
{ {
lock (_gate) lock (_gate)
@@ -695,6 +787,7 @@ public sealed class MemStore : IStreamStore
} }
// Go: memStore.MultiLastSeqs server/memstore.go:828 // Go: memStore.MultiLastSeqs server/memstore.go:828
/// <inheritdoc />
ulong[] IStreamStore.MultiLastSeqs(string[] filters, ulong maxSeq, int maxAllowed) ulong[] IStreamStore.MultiLastSeqs(string[] filters, ulong maxSeq, int maxAllowed)
{ {
lock (_gate) lock (_gate)
@@ -739,6 +832,7 @@ public sealed class MemStore : IStreamStore
} }
// Go: memStore.SubjectForSeq server/memstore.go:1678 // Go: memStore.SubjectForSeq server/memstore.go:1678
/// <inheritdoc />
string IStreamStore.SubjectForSeq(ulong seq) string IStreamStore.SubjectForSeq(ulong seq)
{ {
lock (_gate) lock (_gate)
@@ -750,6 +844,7 @@ public sealed class MemStore : IStreamStore
} }
// Go: memStore.NumPending server/memstore.go:913 // Go: memStore.NumPending server/memstore.go:913
/// <inheritdoc />
(ulong Total, ulong ValidThrough) IStreamStore.NumPending(ulong sseq, string filter, bool lastPerSubject) (ulong Total, ulong ValidThrough) IStreamStore.NumPending(ulong sseq, string filter, bool lastPerSubject)
{ {
lock (_gate) lock (_gate)
@@ -760,6 +855,7 @@ public sealed class MemStore : IStreamStore
} }
// Go: memStore.State server/memstore.go — full state // Go: memStore.State server/memstore.go — full state
/// <inheritdoc />
StorageStreamState IStreamStore.State() StorageStreamState IStreamStore.State()
{ {
lock (_gate) lock (_gate)
@@ -784,6 +880,7 @@ public sealed class MemStore : IStreamStore
} }
// Go: memStore.FastState server/memstore.go — populate without deleted list // Go: memStore.FastState server/memstore.go — populate without deleted list
/// <inheritdoc />
void IStreamStore.FastState(ref StorageStreamState state) void IStreamStore.FastState(ref StorageStreamState state)
{ {
lock (_gate) lock (_gate)
@@ -800,9 +897,11 @@ public sealed class MemStore : IStreamStore
} }
// Go: memStore.Type // Go: memStore.Type
/// <inheritdoc />
StorageType IStreamStore.Type() => StorageType.Memory; StorageType IStreamStore.Type() => StorageType.Memory;
// Go: memStore.UpdateConfig server/memstore.go:86 // Go: memStore.UpdateConfig server/memstore.go:86
/// <inheritdoc />
void IStreamStore.UpdateConfig(StreamConfig cfg) void IStreamStore.UpdateConfig(StreamConfig cfg)
{ {
lock (_gate) lock (_gate)
@@ -831,9 +930,11 @@ public sealed class MemStore : IStreamStore
} }
// Go: memStore.Stop — no-op for in-memory store // Go: memStore.Stop — no-op for in-memory store
/// <inheritdoc />
void IStreamStore.Stop() { } void IStreamStore.Stop() { }
// Go: memStore.Delete — clear everything // Go: memStore.Delete — clear everything
/// <inheritdoc />
void IStreamStore.Delete(bool inline) void IStreamStore.Delete(bool inline)
{ {
lock (_gate) lock (_gate)
@@ -849,11 +950,14 @@ public sealed class MemStore : IStreamStore
} }
// Go: memStore.ResetState // Go: memStore.ResetState
/// <inheritdoc />
void IStreamStore.ResetState() { } void IStreamStore.ResetState() { }
// EncodedStreamState, ConsumerStore — not needed for MemStore tests // EncodedStreamState, ConsumerStore — not needed for MemStore tests
/// <inheritdoc />
byte[] IStreamStore.EncodedStreamState(ulong failed) => []; byte[] IStreamStore.EncodedStreamState(ulong failed) => [];
/// <inheritdoc />
IConsumerStore IStreamStore.ConsumerStore(string name, DateTime created, ConsumerConfig cfg) IConsumerStore IStreamStore.ConsumerStore(string name, DateTime created, ConsumerConfig cfg)
=> throw new NotSupportedException("MemStore does not implement ConsumerStore."); => throw new NotSupportedException("MemStore does not implement ConsumerStore.");
@@ -861,6 +965,10 @@ public sealed class MemStore : IStreamStore
// TrimToMaxMessages — legacy helper used by existing async tests // TrimToMaxMessages — legacy helper used by existing async tests
// ------------------------------------------------------------------------- // -------------------------------------------------------------------------
/// <summary>
/// Trims oldest messages until the stream contains at most <paramref name="maxMessages"/> entries.
/// </summary>
/// <param name="maxMessages">Maximum retained message count.</param>
public void TrimToMaxMessages(ulong maxMessages) public void TrimToMaxMessages(ulong maxMessages)
{ {
lock (_gate) lock (_gate)
@@ -1232,6 +1340,9 @@ public sealed class MemStore : IStreamStore
/// <paramref name="filter"/> at or after <paramref name="start"/>. Called with /// <paramref name="filter"/> at or after <paramref name="start"/>. Called with
/// <c>_gate</c> already held. /// <c>_gate</c> already held.
/// </summary> /// </summary>
/// <param name="filter">Subject wildcard filter used for matching.</param>
/// <param name="start">Minimum sequence to include.</param>
/// <returns>First and last matching sequence with a found flag.</returns>
internal (ulong First, ulong Last, bool Found) NextWildcardMatchLocked(string filter, ulong start) internal (ulong First, ulong Last, bool Found) NextWildcardMatchLocked(string filter, ulong start)
{ {
ulong first = _st.LastSeq, last = 0; ulong first = _st.LastSeq, last = 0;
@@ -1256,6 +1367,9 @@ public sealed class MemStore : IStreamStore
/// equals <paramref name="filter"/> at or after <paramref name="start"/>. Called /// equals <paramref name="filter"/> at or after <paramref name="start"/>. Called
/// with <c>_gate</c> already held. /// with <c>_gate</c> already held.
/// </summary> /// </summary>
/// <param name="filter">Literal subject to match.</param>
/// <param name="start">Minimum sequence to include.</param>
/// <returns>First and last matching sequence with a found flag.</returns>
internal (ulong First, ulong Last, bool Found) NextLiteralMatchLocked(string filter, ulong start) internal (ulong First, ulong Last, bool Found) NextLiteralMatchLocked(string filter, ulong start)
{ {
if (!_fss.TryGetValue(filter, out var ss)) return (0, 0, false); if (!_fss.TryGetValue(filter, out var ss)) return (0, 0, false);
@@ -53,6 +53,7 @@ public sealed class MessageRecord
/// <summary> /// <summary>
/// Encodes a <see cref="MessageRecord"/> to its binary wire format. /// Encodes a <see cref="MessageRecord"/> to its binary wire format.
/// </summary> /// </summary>
/// <param name="record">Record to encode.</param>
/// <returns>The encoded byte array.</returns> /// <returns>The encoded byte array.</returns>
public static byte[] Encode(MessageRecord record) public static byte[] Encode(MessageRecord record)
{ {
@@ -66,6 +67,10 @@ public sealed class MessageRecord
/// <summary> /// <summary>
/// Computes the encoded byte size of a record without allocating. /// Computes the encoded byte size of a record without allocating.
/// </summary> /// </summary>
/// <param name="subject">Subject for the record.</param>
/// <param name="headers">Header bytes for the record.</param>
/// <param name="payload">Payload bytes for the record.</param>
/// <returns>Total encoded byte size.</returns>
public static int MeasureEncodedSize(string subject, ReadOnlySpan<byte> headers, ReadOnlySpan<byte> payload) public static int MeasureEncodedSize(string subject, ReadOnlySpan<byte> headers, ReadOnlySpan<byte> payload)
{ {
var subjectByteCount = Encoding.UTF8.GetByteCount(subject); var subjectByteCount = Encoding.UTF8.GetByteCount(subject);
@@ -81,6 +86,15 @@ public sealed class MessageRecord
/// Go equivalent: writeMsgRecordLocked writes directly into cache.buf. /// Go equivalent: writeMsgRecordLocked writes directly into cache.buf.
/// Returns the number of bytes written. /// Returns the number of bytes written.
/// </summary> /// </summary>
/// <param name="buffer">Target buffer that receives encoded bytes.</param>
/// <param name="bufOffset">Starting offset in <paramref name="buffer"/>.</param>
/// <param name="sequence">Stream sequence to encode.</param>
/// <param name="subject">Subject to encode.</param>
/// <param name="headers">Header bytes to encode.</param>
/// <param name="payload">Payload bytes to encode.</param>
/// <param name="timestamp">Publish timestamp in Unix nanoseconds.</param>
/// <param name="deleted">Whether to mark the record as deleted.</param>
/// <returns>Number of bytes written to <paramref name="buffer"/>.</returns>
public static int EncodeTo( public static int EncodeTo(
byte[] buffer, int bufOffset, byte[] buffer, int bufOffset,
ulong sequence, string subject, ulong sequence, string subject,
@@ -507,6 +507,7 @@ public sealed class MsgBlock : IDisposable
/// This mirrors Go's SkipMsg tombstone behaviour. /// This mirrors Go's SkipMsg tombstone behaviour.
/// Reference: golang/nats-server/server/filestore.go — SkipMsg. /// Reference: golang/nats-server/server/filestore.go — SkipMsg.
/// </summary> /// </summary>
/// <param name="sequence">Sequence number to reserve as a deleted skip record.</param>
public void WriteSkip(ulong sequence) public void WriteSkip(ulong sequence)
{ {
_lock.EnterWriteLock(); _lock.EnterWriteLock();
@@ -647,6 +648,8 @@ public sealed class MsgBlock : IDisposable
/// Returns true if the given sequence number has been soft-deleted in this block. /// Returns true if the given sequence number has been soft-deleted in this block.
/// Reference: golang/nats-server/server/filestore.go — dmap (deleted map) lookup. /// Reference: golang/nats-server/server/filestore.go — dmap (deleted map) lookup.
/// </summary> /// </summary>
/// <param name="sequence">Sequence number to test.</param>
/// <returns><see langword="true"/> when the sequence is marked deleted.</returns>
public bool IsDeleted(ulong sequence) public bool IsDeleted(ulong sequence)
{ {
_lock.EnterReadLock(); _lock.EnterReadLock();
@@ -21,6 +21,8 @@ internal static class S2Codec
/// Returns the compressed bytes, which may be longer than the input for /// Returns the compressed bytes, which may be longer than the input for
/// very small payloads (Snappy does not guarantee compression for tiny inputs). /// very small payloads (Snappy does not guarantee compression for tiny inputs).
/// </summary> /// </summary>
/// <param name="data">Uncompressed payload bytes.</param>
/// <returns>Compressed payload bytes.</returns>
public static byte[] Compress(ReadOnlySpan<byte> data) public static byte[] Compress(ReadOnlySpan<byte> data)
{ {
if (data.IsEmpty) if (data.IsEmpty)
@@ -32,6 +34,8 @@ internal static class S2Codec
/// <summary> /// <summary>
/// Decompresses Snappy-compressed <paramref name="data"/>. /// Decompresses Snappy-compressed <paramref name="data"/>.
/// </summary> /// </summary>
/// <param name="data">Compressed payload bytes.</param>
/// <returns>Decompressed payload bytes.</returns>
/// <exception cref="InvalidDataException">If the data is not valid Snappy.</exception> /// <exception cref="InvalidDataException">If the data is not valid Snappy.</exception>
public static byte[] Decompress(ReadOnlySpan<byte> data) public static byte[] Decompress(ReadOnlySpan<byte> data)
{ {
@@ -45,6 +49,9 @@ internal static class S2Codec
/// Compresses only the body portion of <paramref name="data"/>, leaving the /// Compresses only the body portion of <paramref name="data"/>, leaving the
/// last <paramref name="checksumSize"/> bytes uncompressed (appended verbatim). /// last <paramref name="checksumSize"/> bytes uncompressed (appended verbatim).
/// </summary> /// </summary>
/// <param name="data">Body plus trailing checksum bytes.</param>
/// <param name="checksumSize">Number of trailing checksum bytes to keep raw.</param>
/// <returns>Compressed body with raw trailing checksum bytes appended.</returns>
/// <remarks> /// <remarks>
/// In the Go FileStore the trailing bytes of a stored record can be a raw /// In the Go FileStore the trailing bytes of a stored record can be a raw
/// checksum that is not part of the compressed payload. This helper mirrors /// checksum that is not part of the compressed payload. This helper mirrors
@@ -82,6 +89,9 @@ internal static class S2Codec
/// Decompresses only the body portion of <paramref name="data"/>, treating /// Decompresses only the body portion of <paramref name="data"/>, treating
/// the last <paramref name="checksumSize"/> bytes as a raw (uncompressed) checksum. /// the last <paramref name="checksumSize"/> bytes as a raw (uncompressed) checksum.
/// </summary> /// </summary>
/// <param name="data">Compressed body plus trailing checksum bytes.</param>
/// <param name="checksumSize">Number of trailing checksum bytes kept raw.</param>
/// <returns>Decompressed body with original trailing checksum bytes appended.</returns>
public static byte[] DecompressWithTrailingChecksum(ReadOnlySpan<byte> data, int checksumSize) public static byte[] DecompressWithTrailingChecksum(ReadOnlySpan<byte> data, int checksumSize)
{ {
if (checksumSize < 0) if (checksumSize < 0)
@@ -48,6 +48,8 @@ internal sealed class SequenceSet : IEnumerable<ulong>
/// Returns <c>true</c> if the sequence was not already present. /// Returns <c>true</c> if the sequence was not already present.
/// Reference: golang/nats-server/server/avl/seqset.go:44 (Insert). /// Reference: golang/nats-server/server/avl/seqset.go:44 (Insert).
/// </summary> /// </summary>
/// <param name="seq">Sequence to add.</param>
/// <returns><see langword="true"/> when the sequence was newly added.</returns>
public bool Add(ulong seq) public bool Add(ulong seq)
{ {
// Strategy: find the position where seq belongs (binary search by Start), // Strategy: find the position where seq belongs (binary search by Start),
@@ -122,6 +124,8 @@ internal sealed class SequenceSet : IEnumerable<ulong>
/// Returns <c>true</c> if the sequence was present. /// Returns <c>true</c> if the sequence was present.
/// Reference: golang/nats-server/server/avl/seqset.go:80 (Delete). /// Reference: golang/nats-server/server/avl/seqset.go:80 (Delete).
/// </summary> /// </summary>
/// <param name="seq">Sequence to remove.</param>
/// <returns><see langword="true"/> when the sequence existed in the set.</returns>
public bool Remove(ulong seq) public bool Remove(ulong seq)
{ {
// Binary search for the range that contains seq. // Binary search for the range that contains seq.
@@ -170,6 +174,8 @@ internal sealed class SequenceSet : IEnumerable<ulong>
/// Binary search: O(log R) where R is the number of distinct ranges. /// Binary search: O(log R) where R is the number of distinct ranges.
/// Reference: golang/nats-server/server/avl/seqset.go:52 (Exists). /// Reference: golang/nats-server/server/avl/seqset.go:52 (Exists).
/// </summary> /// </summary>
/// <param name="seq">Sequence to test for membership.</param>
/// <returns><see langword="true"/> when the set contains <paramref name="seq"/>.</returns>
public bool Contains(ulong seq) public bool Contains(ulong seq)
{ {
var lo = 0; var lo = 0;
@@ -225,6 +231,7 @@ internal sealed class SequenceSet : IEnumerable<ulong>
} }
} }
/// <inheritdoc />
System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator() System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
=> GetEnumerator(); => GetEnumerator();
} }
@@ -10,18 +10,23 @@ namespace NATS.Server.JetStream.Storage;
public sealed class StoreMsg public sealed class StoreMsg
{ {
// Go: StoreMsg.subj // Go: StoreMsg.subj
/// <summary>Subject associated with this stored message.</summary>
public string Subject { get; set; } = string.Empty; public string Subject { get; set; } = string.Empty;
// Go: StoreMsg.hdr — NATS message headers (optional) // Go: StoreMsg.hdr — NATS message headers (optional)
/// <summary>Optional encoded header bytes.</summary>
public byte[]? Header { get; set; } public byte[]? Header { get; set; }
// Go: StoreMsg.msg — message body // Go: StoreMsg.msg — message body
/// <summary>Optional message payload bytes.</summary>
public byte[]? Data { get; set; } public byte[]? Data { get; set; }
// Go: StoreMsg.seq — stream sequence number // Go: StoreMsg.seq — stream sequence number
/// <summary>Stream sequence number.</summary>
public ulong Sequence { get; set; } public ulong Sequence { get; set; }
// Go: StoreMsg.ts — wall-clock timestamp in Unix nanoseconds // Go: StoreMsg.ts — wall-clock timestamp in Unix nanoseconds
/// <summary>Publish timestamp in Unix nanoseconds.</summary>
public long Timestamp { get; set; } public long Timestamp { get; set; }
/// <summary> /// <summary>
@@ -2,12 +2,19 @@ namespace NATS.Server.JetStream.Storage;
public sealed class StoredMessage public sealed class StoredMessage
{ {
/// <summary>Stream sequence assigned to this message.</summary>
public ulong Sequence { get; init; } public ulong Sequence { get; init; }
/// <summary>Subject the message was published to.</summary>
public string Subject { get; init; } = string.Empty; public string Subject { get; init; } = string.Empty;
/// <summary>Message payload bytes.</summary>
public ReadOnlyMemory<byte> Payload { get; init; } public ReadOnlyMemory<byte> Payload { get; init; }
/// <summary>Raw protocol header bytes used for header parsing and replay.</summary>
internal ReadOnlyMemory<byte> RawHeaders { get; init; } internal ReadOnlyMemory<byte> RawHeaders { get; init; }
/// <summary>Message timestamp in UTC.</summary>
public DateTime TimestampUtc { get; init; } = DateTime.UtcNow; public DateTime TimestampUtc { get; init; } = DateTime.UtcNow;
/// <summary>Optional account name associated with the message.</summary>
public string? Account { get; init; } public string? Account { get; init; }
/// <summary>Indicates whether the message has been redelivered.</summary>
public bool Redelivered { get; init; } public bool Redelivered { get; init; }
/// <summary> /// <summary>
@@ -20,6 +27,10 @@ public sealed class StoredMessage
/// </summary> /// </summary>
public string? MsgId => Headers is not null && Headers.TryGetValue("Nats-Msg-Id", out var id) ? id : null; public string? MsgId => Headers is not null && Headers.TryGetValue("Nats-Msg-Id", out var id) ? id : null;
/// <summary>
/// Converts this message to a compact index representation.
/// </summary>
/// <returns>Message index used by listing and lookup operations.</returns>
internal StoredMessageIndex ToIndex() internal StoredMessageIndex ToIndex()
=> new(Sequence, Subject, Payload.Length, TimestampUtc); => new(Sequence, Subject, Payload.Length, TimestampUtc);
} }
@@ -9,39 +9,51 @@ namespace NATS.Server.JetStream.Storage;
public record struct StreamState public record struct StreamState
{ {
// Go: StreamState.Msgs — total number of messages in the stream // Go: StreamState.Msgs — total number of messages in the stream
/// <summary>Total number of retained messages in the stream.</summary>
public ulong Msgs { get; set; } public ulong Msgs { get; set; }
// Go: StreamState.Bytes — total bytes stored // Go: StreamState.Bytes — total bytes stored
/// <summary>Total bytes consumed by retained messages.</summary>
public ulong Bytes { get; set; } public ulong Bytes { get; set; }
// Go: StreamState.FirstSeq — sequence number of the oldest message // Go: StreamState.FirstSeq — sequence number of the oldest message
/// <summary>Sequence number of the oldest retained message.</summary>
public ulong FirstSeq { get; set; } public ulong FirstSeq { get; set; }
// Go: StreamState.FirstTime — wall-clock time of the oldest message // Go: StreamState.FirstTime — wall-clock time of the oldest message
/// <summary>Timestamp of the oldest retained message.</summary>
public DateTime FirstTime { get; set; } public DateTime FirstTime { get; set; }
// Go: StreamState.LastSeq — sequence number of the newest message // Go: StreamState.LastSeq — sequence number of the newest message
/// <summary>Sequence number of the newest retained message.</summary>
public ulong LastSeq { get; set; } public ulong LastSeq { get; set; }
// Go: StreamState.LastTime — wall-clock time of the newest message // Go: StreamState.LastTime — wall-clock time of the newest message
/// <summary>Timestamp of the newest retained message.</summary>
public DateTime LastTime { get; set; } public DateTime LastTime { get; set; }
// Go: StreamState.NumSubjects — count of distinct subjects in the stream // Go: StreamState.NumSubjects — count of distinct subjects in the stream
/// <summary>Count of distinct subjects currently represented in the stream.</summary>
public int NumSubjects { get; set; } public int NumSubjects { get; set; }
// Go: StreamState.Subjects — per-subject message counts (populated on demand) // Go: StreamState.Subjects — per-subject message counts (populated on demand)
/// <summary>Optional per-subject retained message totals.</summary>
public Dictionary<string, ulong>? Subjects { get; set; } public Dictionary<string, ulong>? Subjects { get; set; }
// Go: StreamState.NumDeleted — number of interior gaps (deleted sequences) // Go: StreamState.NumDeleted — number of interior gaps (deleted sequences)
/// <summary>Count of deleted interior sequences currently tracked.</summary>
public int NumDeleted { get; set; } public int NumDeleted { get; set; }
// Go: StreamState.Deleted — explicit list of deleted sequences (populated on demand) // Go: StreamState.Deleted — explicit list of deleted sequences (populated on demand)
/// <summary>Optional list of deleted interior sequence numbers.</summary>
public ulong[]? Deleted { get; set; } public ulong[]? Deleted { get; set; }
// Go: StreamState.Lost (LostStreamData) — sequences that were lost due to storage corruption // Go: StreamState.Lost (LostStreamData) — sequences that were lost due to storage corruption
/// <summary>Optional corruption/loss metadata for this stream.</summary>
public LostStreamData? Lost { get; set; } public LostStreamData? Lost { get; set; }
// Go: StreamState.Consumers — number of consumers attached to the stream // Go: StreamState.Consumers — number of consumers attached to the stream
/// <summary>Number of consumers currently attached to the stream.</summary>
public int Consumers { get; set; } public int Consumers { get; set; }
} }
@@ -53,9 +65,11 @@ public record struct StreamState
public sealed class LostStreamData public sealed class LostStreamData
{ {
// Go: LostStreamData.Msgs — sequences of lost messages // Go: LostStreamData.Msgs — sequences of lost messages
/// <summary>Sequences that were lost due to storage corruption.</summary>
public ulong[]? Msgs { get; set; } public ulong[]? Msgs { get; set; }
// Go: LostStreamData.Bytes — total bytes of lost data // Go: LostStreamData.Bytes — total bytes of lost data
/// <summary>Total bytes estimated as lost.</summary>
public ulong Bytes { get; set; } public ulong Bytes { get; set; }
} }
@@ -68,11 +82,14 @@ public sealed class LostStreamData
public record struct SimpleState public record struct SimpleState
{ {
// Go: SimpleState.Msgs — number of messages matching the filter // Go: SimpleState.Msgs — number of messages matching the filter
/// <summary>Count of matching retained messages.</summary>
public ulong Msgs { get; set; } public ulong Msgs { get; set; }
// Go: SimpleState.First — first sequence number matching the filter // Go: SimpleState.First — first sequence number matching the filter
/// <summary>First sequence that matches the filter.</summary>
public ulong First { get; set; } public ulong First { get; set; }
// Go: SimpleState.Last — last sequence number matching the filter // Go: SimpleState.Last — last sequence number matching the filter
/// <summary>Last sequence that matches the filter.</summary>
public ulong Last { get; set; } public ulong Last { get; set; }
} }
@@ -20,9 +20,13 @@ public sealed class LeafConnection(Socket socket) : IAsyncDisposable
private string? _remoteCluster; private string? _remoteCluster;
private Task? _loopTask; private Task? _loopTask;
/// <summary>Remote server identifier learned from LEAF handshake.</summary>
public string? RemoteId { get; internal set; } public string? RemoteId { get; internal set; }
/// <summary>Remote endpoint string for diagnostics and monitoring.</summary>
public string RemoteEndpoint => socket.RemoteEndPoint?.ToString() ?? Guid.NewGuid().ToString("N"); public string RemoteEndpoint => socket.RemoteEndPoint?.ToString() ?? Guid.NewGuid().ToString("N");
/// <summary>Callback invoked when remote LS+/LS- interest updates are received.</summary>
public Func<RemoteSubscription, Task>? RemoteSubscriptionReceived { get; set; } public Func<RemoteSubscription, Task>? RemoteSubscriptionReceived { get; set; }
/// <summary>Callback invoked when remote LMSG payloads are received.</summary>
public Func<LeafMessage, Task>? MessageReceived { get; set; } public Func<LeafMessage, Task>? MessageReceived { get; set; }
/// <summary> /// <summary>
@@ -97,6 +101,8 @@ public sealed class LeafConnection(Socket socket) : IAsyncDisposable
/// permissions as synced. Passing null for either list clears that list. /// permissions as synced. Passing null for either list clears that list.
/// Go reference: leafnode.go — sendPermsAndAccountInfo. /// Go reference: leafnode.go — sendPermsAndAccountInfo.
/// </summary> /// </summary>
/// <param name="publishAllow">Subjects this leaf is allowed to publish to.</param>
/// <param name="subscribeAllow">Subjects this leaf is allowed to subscribe to.</param>
public void SetPermissions(IEnumerable<string>? publishAllow, IEnumerable<string>? subscribeAllow) public void SetPermissions(IEnumerable<string>? publishAllow, IEnumerable<string>? subscribeAllow)
{ {
AllowedPublishSubjects.Clear(); AllowedPublishSubjects.Clear();
@@ -111,6 +117,11 @@ public sealed class LeafConnection(Socket socket) : IAsyncDisposable
PermsSynced = true; PermsSynced = true;
} }
/// <summary>
/// Performs the outbound LEAF handshake for a solicited connection.
/// </summary>
/// <param name="serverId">Local server identifier to advertise.</param>
/// <param name="ct">Cancellation token for I/O operations.</param>
public async Task PerformOutboundHandshakeAsync(string serverId, CancellationToken ct) public async Task PerformOutboundHandshakeAsync(string serverId, CancellationToken ct)
{ {
var handshakeLine = BuildHandshakeLine(serverId); var handshakeLine = BuildHandshakeLine(serverId);
@@ -119,6 +130,11 @@ public sealed class LeafConnection(Socket socket) : IAsyncDisposable
ParseHandshakeResponse(line); ParseHandshakeResponse(line);
} }
/// <summary>
/// Performs the inbound LEAF handshake for an accepted connection.
/// </summary>
/// <param name="serverId">Local server identifier to advertise.</param>
/// <param name="ct">Cancellation token for I/O operations.</param>
public async Task PerformInboundHandshakeAsync(string serverId, CancellationToken ct) public async Task PerformInboundHandshakeAsync(string serverId, CancellationToken ct)
{ {
var line = await ReadLineAsync(ct); var line = await ReadLineAsync(ct);
@@ -127,6 +143,10 @@ public sealed class LeafConnection(Socket socket) : IAsyncDisposable
await WriteLineAsync(handshakeLine, ct); await WriteLineAsync(handshakeLine, ct);
} }
/// <summary>
/// Starts the background read loop for this leaf connection.
/// </summary>
/// <param name="ct">Cancellation token controlling the loop lifetime.</param>
public void StartLoop(CancellationToken ct) public void StartLoop(CancellationToken ct)
{ {
if (_loopTask != null) if (_loopTask != null)
@@ -136,12 +156,32 @@ public sealed class LeafConnection(Socket socket) : IAsyncDisposable
_loopTask = Task.Run(() => ReadLoopAsync(linked.Token), linked.Token); _loopTask = Task.Run(() => ReadLoopAsync(linked.Token), linked.Token);
} }
/// <summary>
/// Waits until the leaf read loop exits.
/// </summary>
/// <param name="ct">Cancellation token used while waiting.</param>
/// <returns>A task that completes when the loop is closed.</returns>
public Task WaitUntilClosedAsync(CancellationToken ct) public Task WaitUntilClosedAsync(CancellationToken ct)
=> _loopTask?.WaitAsync(ct) ?? Task.CompletedTask; => _loopTask?.WaitAsync(ct) ?? Task.CompletedTask;
/// <summary>
/// Sends LS+ interest for a subject, optionally with queue group.
/// </summary>
/// <param name="account">Account for the interest update.</param>
/// <param name="subject">Subject being added.</param>
/// <param name="queue">Optional queue group name.</param>
/// <param name="ct">Cancellation token for I/O operations.</param>
public Task SendLsPlusAsync(string account, string subject, string? queue, CancellationToken ct) public Task SendLsPlusAsync(string account, string subject, string? queue, CancellationToken ct)
=> SendLsPlusAsync(account, subject, queue, queueWeight: 0, ct); => SendLsPlusAsync(account, subject, queue, queueWeight: 0, ct);
/// <summary>
/// Sends LS+ interest for a subject with optional queue group and weight.
/// </summary>
/// <param name="account">Account for the interest update.</param>
/// <param name="subject">Subject being added.</param>
/// <param name="queue">Optional queue group name.</param>
/// <param name="queueWeight">Queue weight to advertise when queue is present.</param>
/// <param name="ct">Cancellation token for I/O operations.</param>
public Task SendLsPlusAsync(string account, string subject, string? queue, int queueWeight, CancellationToken ct) public Task SendLsPlusAsync(string account, string subject, string? queue, int queueWeight, CancellationToken ct)
{ {
string frame; string frame;
@@ -155,6 +195,13 @@ public sealed class LeafConnection(Socket socket) : IAsyncDisposable
return WriteLineAsync(frame, ct); return WriteLineAsync(frame, ct);
} }
/// <summary>
/// Sends LS- interest removal for a subject.
/// </summary>
/// <param name="account">Account for the interest update.</param>
/// <param name="subject">Subject being removed.</param>
/// <param name="queue">Optional queue group name.</param>
/// <param name="ct">Cancellation token for I/O operations.</param>
public Task SendLsMinusAsync(string account, string subject, string? queue, CancellationToken ct) public Task SendLsMinusAsync(string account, string subject, string? queue, CancellationToken ct)
=> WriteLineAsync(queue is { Length: > 0 } ? $"LS- {account} {subject} {queue}" : $"LS- {account} {subject}", ct); => WriteLineAsync(queue is { Length: > 0 } ? $"LS- {account} {subject} {queue}" : $"LS- {account} {subject}", ct);
@@ -162,6 +209,8 @@ public sealed class LeafConnection(Socket socket) : IAsyncDisposable
/// Sends a CONNECT protocol line with JSON payload for solicited leaf links. /// Sends a CONNECT protocol line with JSON payload for solicited leaf links.
/// Go reference: leafnode.go sendLeafConnect. /// Go reference: leafnode.go sendLeafConnect.
/// </summary> /// </summary>
/// <param name="connectInfo">Leaf CONNECT payload to serialize.</param>
/// <param name="ct">Cancellation token for I/O operations.</param>
public Task SendLeafConnectAsync(LeafConnectInfo connectInfo, CancellationToken ct) public Task SendLeafConnectAsync(LeafConnectInfo connectInfo, CancellationToken ct)
{ {
ArgumentNullException.ThrowIfNull(connectInfo); ArgumentNullException.ThrowIfNull(connectInfo);
@@ -169,6 +218,14 @@ public sealed class LeafConnection(Socket socket) : IAsyncDisposable
return WriteLineAsync($"CONNECT {json}", ct); return WriteLineAsync($"CONNECT {json}", ct);
} }
/// <summary>
/// Sends an LMSG frame to the remote leaf connection.
/// </summary>
/// <param name="account">Account associated with the message.</param>
/// <param name="subject">Subject to deliver.</param>
/// <param name="replyTo">Optional reply subject.</param>
/// <param name="payload">Payload bytes.</param>
/// <param name="ct">Cancellation token for I/O operations.</param>
public async Task SendMessageAsync(string account, string subject, string? replyTo, ReadOnlyMemory<byte> payload, CancellationToken ct) public async Task SendMessageAsync(string account, string subject, string? replyTo, ReadOnlyMemory<byte> payload, CancellationToken ct)
{ {
var reply = string.IsNullOrEmpty(replyTo) ? "-" : replyTo; var reply = string.IsNullOrEmpty(replyTo) ? "-" : replyTo;
@@ -188,6 +245,9 @@ public sealed class LeafConnection(Socket socket) : IAsyncDisposable
} }
} }
/// <summary>
/// Disposes this leaf connection and stops background processing.
/// </summary>
public async ValueTask DisposeAsync() public async ValueTask DisposeAsync()
{ {
await _closedCts.CancelAsync(); await _closedCts.CancelAsync();
@@ -206,16 +266,22 @@ public sealed class LeafConnection(Socket socket) : IAsyncDisposable
return $"LEAF {serverId}"; return $"LEAF {serverId}";
} }
/// <summary>Indicates whether this is a solicited leaf connection.</summary>
public bool IsSolicitedLeafNode() => IsSolicited; public bool IsSolicitedLeafNode() => IsSolicited;
/// <summary>Indicates whether this leaf is operating in spoke mode.</summary>
public bool IsSpokeLeafNode() => IsSpoke; public bool IsSpokeLeafNode() => IsSpoke;
/// <summary>Indicates whether this leaf is operating in hub mode.</summary>
public bool IsHubLeafNode() => !IsSpoke; public bool IsHubLeafNode() => !IsSpoke;
/// <summary>Indicates whether this leaf is isolated from hub propagation.</summary>
public bool IsIsolatedLeafNode() => Isolated; public bool IsIsolatedLeafNode() => Isolated;
/// <summary>Returns the remote cluster name if advertised by the peer.</summary>
public string? RemoteCluster() => _remoteCluster; public string? RemoteCluster() => _remoteCluster;
/// <summary> /// <summary>
/// Applies connect delay only when this is a solicited leaf connection. /// Applies connect delay only when this is a solicited leaf connection.
/// Go reference: leafnode.go setLeafConnectDelayIfSoliciting. /// Go reference: leafnode.go setLeafConnectDelayIfSoliciting.
/// </summary> /// </summary>
/// <param name="delay">Reconnect delay to apply.</param>
public void SetLeafConnectDelayIfSoliciting(TimeSpan delay) public void SetLeafConnectDelayIfSoliciting(TimeSpan delay)
{ {
if (IsSolicited) if (IsSolicited)
@@ -226,6 +292,7 @@ public sealed class LeafConnection(Socket socket) : IAsyncDisposable
/// Handles remote ERR protocol for leaf links and applies reconnect delay hints. /// Handles remote ERR protocol for leaf links and applies reconnect delay hints.
/// Go reference: leafnode.go leafProcessErr. /// Go reference: leafnode.go leafProcessErr.
/// </summary> /// </summary>
/// <param name="errStr">Error text received from the remote leaf peer.</param>
public void LeafProcessErr(string errStr) public void LeafProcessErr(string errStr)
{ {
if (string.IsNullOrWhiteSpace(errStr)) if (string.IsNullOrWhiteSpace(errStr))
@@ -254,12 +321,15 @@ public sealed class LeafConnection(Socket socket) : IAsyncDisposable
/// Handles subscription permission violations. /// Handles subscription permission violations.
/// Go reference: leafnode.go leafSubPermViolation. /// Go reference: leafnode.go leafSubPermViolation.
/// </summary> /// </summary>
/// <param name="subj">Subject that triggered the violation.</param>
public void LeafSubPermViolation(string subj) => LeafPermViolation(pub: false, subj); public void LeafSubPermViolation(string subj) => LeafPermViolation(pub: false, subj);
/// <summary> /// <summary>
/// Handles publish/subscribe permission violations. /// Handles publish/subscribe permission violations.
/// Go reference: leafnode.go leafPermViolation. /// Go reference: leafnode.go leafPermViolation.
/// </summary> /// </summary>
/// <param name="pub"><see langword="true"/> for publish violations, otherwise subscribe.</param>
/// <param name="subj">Subject that triggered the violation.</param>
public void LeafPermViolation(bool pub, string subj) public void LeafPermViolation(bool pub, string subj)
=> SetLeafConnectDelayIfSoliciting(LeafNodeManager.LeafNodeReconnectAfterPermViolation); => SetLeafConnectDelayIfSoliciting(LeafNodeManager.LeafNodeReconnectAfterPermViolation);
@@ -26,8 +26,11 @@ public sealed class WebSocketStreamAdapter : Stream
} }
// Stream capability overrides // Stream capability overrides
/// <inheritdoc />
public override bool CanRead => true; public override bool CanRead => true;
/// <inheritdoc />
public override bool CanWrite => true; public override bool CanWrite => true;
/// <inheritdoc />
public override bool CanSeek => false; public override bool CanSeek => false;
// Telemetry properties // Telemetry properties
@@ -37,12 +40,7 @@ public sealed class WebSocketStreamAdapter : Stream
public int MessagesRead { get; private set; } public int MessagesRead { get; private set; }
public int MessagesWritten { get; private set; } public int MessagesWritten { get; private set; }
/// <summary> /// <inheritdoc />
/// Reads data from the WebSocket into <paramref name="buffer"/>.
/// If the internal read buffer has buffered data from a previous message,
/// that is served first. Otherwise a new WebSocket message is received.
/// Go reference: client.go wsRead.
/// </summary>
public override async Task<int> ReadAsync(byte[] buffer, int offset, int count, CancellationToken ct) public override async Task<int> ReadAsync(byte[] buffer, int offset, int count, CancellationToken ct)
{ {
ObjectDisposedException.ThrowIf(_disposed, this); ObjectDisposedException.ThrowIf(_disposed, this);
@@ -160,10 +158,7 @@ public sealed class WebSocketStreamAdapter : Stream
} }
} }
/// <summary> /// <inheritdoc />
/// Sends <paramref name="buffer"/> as a single binary WebSocket message.
/// Go reference: client.go wsWrite.
/// </summary>
public override async Task WriteAsync(byte[] buffer, int offset, int count, CancellationToken ct) public override async Task WriteAsync(byte[] buffer, int offset, int count, CancellationToken ct)
{ {
ObjectDisposedException.ThrowIf(_disposed, this); ObjectDisposedException.ThrowIf(_disposed, this);
@@ -193,7 +188,9 @@ public sealed class WebSocketStreamAdapter : Stream
public override Task FlushAsync(CancellationToken ct) => Task.CompletedTask; public override Task FlushAsync(CancellationToken ct) => Task.CompletedTask;
// Not-supported synchronous and seeking members // Not-supported synchronous and seeking members
/// <inheritdoc />
public override long Length => throw new NotSupportedException(); public override long Length => throw new NotSupportedException();
/// <inheritdoc />
public override long Position public override long Position
{ {
get => throw new NotSupportedException(); get => throw new NotSupportedException();
+32
View File
@@ -47,11 +47,15 @@ public sealed class MqttConnection : IAsyncDisposable
/// </summary> /// </summary>
public MqttNatsClientAdapter? Adapter { get; private set; } public MqttNatsClientAdapter? Adapter { get; private set; }
/// <summary>MQTT client identifier currently bound to this connection.</summary>
public string ClientId => _clientId; public string ClientId => _clientId;
/// <summary> /// <summary>
/// Creates a connection from a TcpClient (standard accept path). /// Creates a connection from a TcpClient (standard accept path).
/// </summary> /// </summary>
/// <param name="client">Accepted TCP client transport.</param>
/// <param name="listener">Owning MQTT listener and session coordinator.</param>
/// <param name="useBinaryProtocol">Whether to run MQTT binary protocol mode.</param>
public MqttConnection(TcpClient client, MqttListener listener, bool useBinaryProtocol = true) public MqttConnection(TcpClient client, MqttListener listener, bool useBinaryProtocol = true)
{ {
_tcpClient = client; _tcpClient = client;
@@ -64,6 +68,9 @@ public sealed class MqttConnection : IAsyncDisposable
/// <summary> /// <summary>
/// Creates a connection from an arbitrary Stream (for TLS wrapping or testing). /// Creates a connection from an arbitrary Stream (for TLS wrapping or testing).
/// </summary> /// </summary>
/// <param name="stream">Input/output transport stream for this connection.</param>
/// <param name="listener">Owning MQTT listener and session coordinator.</param>
/// <param name="useBinaryProtocol">Whether to run MQTT binary protocol mode.</param>
public MqttConnection(Stream stream, MqttListener listener, bool useBinaryProtocol = true) public MqttConnection(Stream stream, MqttListener listener, bool useBinaryProtocol = true)
{ {
_stream = stream; _stream = stream;
@@ -76,6 +83,10 @@ public sealed class MqttConnection : IAsyncDisposable
/// Creates a connection from a Stream with a TLS client certificate. /// Creates a connection from a Stream with a TLS client certificate.
/// Used by the accept loop after TLS handshake completes. /// Used by the accept loop after TLS handshake completes.
/// </summary> /// </summary>
/// <param name="stream">Input/output transport stream for this connection.</param>
/// <param name="listener">Owning MQTT listener and session coordinator.</param>
/// <param name="useBinaryProtocol">Whether to run MQTT binary protocol mode.</param>
/// <param name="clientCert">Peer certificate captured during TLS handshake.</param>
public MqttConnection(Stream stream, MqttListener listener, bool useBinaryProtocol, X509Certificate2? clientCert) public MqttConnection(Stream stream, MqttListener listener, bool useBinaryProtocol, X509Certificate2? clientCert)
{ {
_stream = stream; _stream = stream;
@@ -85,6 +96,10 @@ public sealed class MqttConnection : IAsyncDisposable
_isPlainSocket = false; // TLS-wrapped stream _isPlainSocket = false; // TLS-wrapped stream
} }
/// <summary>
/// Runs the MQTT connection loop until cancellation or disconnect.
/// </summary>
/// <param name="ct">Cancellation token controlling connection lifetime.</param>
public async Task RunAsync(CancellationToken ct) public async Task RunAsync(CancellationToken ct)
{ {
if (_useBinaryProtocol) if (_useBinaryProtocol)
@@ -492,6 +507,12 @@ public sealed class MqttConnection : IAsyncDisposable
/// <summary> /// <summary>
/// Sends a binary MQTT PUBLISH packet to this connection (for message delivery). /// Sends a binary MQTT PUBLISH packet to this connection (for message delivery).
/// </summary> /// </summary>
/// <param name="topic">Topic to publish to the client.</param>
/// <param name="payload">Message payload bytes.</param>
/// <param name="qos">MQTT QoS level for delivery.</param>
/// <param name="retain">Whether the retained flag should be set on the packet.</param>
/// <param name="packetId">Packet identifier for QoS flows that require acknowledgements.</param>
/// <param name="ct">Cancellation token for the send operation.</param>
public async Task SendBinaryPublishAsync(string topic, ReadOnlyMemory<byte> payload, byte qos, public async Task SendBinaryPublishAsync(string topic, ReadOnlyMemory<byte> payload, byte qos,
bool retain, ushort packetId, CancellationToken ct) bool retain, ushort packetId, CancellationToken ct)
{ {
@@ -503,6 +524,9 @@ public sealed class MqttConnection : IAsyncDisposable
/// Sends a message to the connection. Used by the listener for fan-out delivery. /// Sends a message to the connection. Used by the listener for fan-out delivery.
/// In binary mode, sends a PUBLISH packet; in text mode, sends a text line. /// In binary mode, sends a PUBLISH packet; in text mode, sends a text line.
/// </summary> /// </summary>
/// <param name="topic">Destination topic.</param>
/// <param name="payload">UTF-8 payload text.</param>
/// <param name="ct">Cancellation token for the send operation.</param>
public Task SendMessageAsync(string topic, string payload, CancellationToken ct) public Task SendMessageAsync(string topic, string payload, CancellationToken ct)
{ {
if (_useBinaryProtocol) if (_useBinaryProtocol)
@@ -519,6 +543,11 @@ public sealed class MqttConnection : IAsyncDisposable
/// Zero-allocation hot path — formats the packet directly into the buffer. /// Zero-allocation hot path — formats the packet directly into the buffer.
/// Called synchronously from the NATS delivery path (DeliverMessage). /// Called synchronously from the NATS delivery path (DeliverMessage).
/// </summary> /// </summary>
/// <param name="topicUtf8">Destination topic encoded as UTF-8 bytes.</param>
/// <param name="payload">Message payload bytes.</param>
/// <param name="qos">MQTT QoS level for the packet.</param>
/// <param name="retain">Whether to set the retain flag.</param>
/// <param name="packetId">Packet identifier used for QoS handshakes.</param>
public void EnqueuePublishNoFlush(ReadOnlySpan<byte> topicUtf8, ReadOnlyMemory<byte> payload, public void EnqueuePublishNoFlush(ReadOnlySpan<byte> topicUtf8, ReadOnlyMemory<byte> payload,
byte qos = 0, bool retain = false, ushort packetId = 0) byte qos = 0, bool retain = false, ushort packetId = 0)
{ {
@@ -606,6 +635,9 @@ public sealed class MqttConnection : IAsyncDisposable
catch (ObjectDisposedException) { } catch (ObjectDisposedException) { }
} }
/// <summary>
/// Disposes connection resources and unregisters listener/adapter state.
/// </summary>
public async ValueTask DisposeAsync() public async ValueTask DisposeAsync()
{ {
// Clean up adapter subscriptions and unregister from listener // Clean up adapter subscriptions and unregister from listener
@@ -25,6 +25,11 @@ public sealed class MqttConsumerManager
private readonly ConsumerManager _consumerManager; private readonly ConsumerManager _consumerManager;
private readonly ConcurrentDictionary<string, MqttConsumerBinding> _bindings = new(StringComparer.Ordinal); private readonly ConcurrentDictionary<string, MqttConsumerBinding> _bindings = new(StringComparer.Ordinal);
/// <summary>
/// Creates an MQTT consumer manager backed by JetStream stream and consumer managers.
/// </summary>
/// <param name="streamManager">Stream manager used to resolve MQTT backing streams.</param>
/// <param name="consumerManager">Consumer manager used to create and delete durable consumers.</param>
public MqttConsumerManager(StreamManager streamManager, ConsumerManager consumerManager) public MqttConsumerManager(StreamManager streamManager, ConsumerManager consumerManager)
{ {
_streamManager = streamManager; _streamManager = streamManager;
@@ -37,6 +42,11 @@ public sealed class MqttConsumerManager
/// Returns the binding, or null if creation failed. /// Returns the binding, or null if creation failed.
/// Go reference: server/mqtt.go mqttProcessSub consumer creation. /// Go reference: server/mqtt.go mqttProcessSub consumer creation.
/// </summary> /// </summary>
/// <param name="clientId">MQTT client identifier.</param>
/// <param name="natsSubject">NATS subject mapped from the MQTT topic filter.</param>
/// <param name="qos">Requested MQTT QoS level.</param>
/// <param name="maxAckPending">Maximum number of unacknowledged deliveries.</param>
/// <returns>Created consumer binding, or <see langword="null"/> on failure.</returns>
public MqttConsumerBinding? CreateSubscriptionConsumer(string clientId, string natsSubject, int qos, int maxAckPending) public MqttConsumerBinding? CreateSubscriptionConsumer(string clientId, string natsSubject, int qos, int maxAckPending)
{ {
var durableName = $"$MQTT_{clientId}_{natsSubject.Replace('.', '_').Replace('*', 'W').Replace('>', 'G')}"; var durableName = $"$MQTT_{clientId}_{natsSubject.Replace('.', '_').Replace('*', 'W').Replace('>', 'G')}";
@@ -65,6 +75,8 @@ public sealed class MqttConsumerManager
/// Removes the JetStream consumer for an MQTT subscription. /// Removes the JetStream consumer for an MQTT subscription.
/// Called on UNSUBSCRIBE or clean session disconnect. /// Called on UNSUBSCRIBE or clean session disconnect.
/// </summary> /// </summary>
/// <param name="clientId">MQTT client identifier.</param>
/// <param name="natsSubject">NATS subject mapped from the MQTT topic filter.</param>
public void RemoveSubscriptionConsumer(string clientId, string natsSubject) public void RemoveSubscriptionConsumer(string clientId, string natsSubject)
{ {
var key = $"{clientId}:{natsSubject}"; var key = $"{clientId}:{natsSubject}";
@@ -77,6 +89,7 @@ public sealed class MqttConsumerManager
/// <summary> /// <summary>
/// Removes all consumers for a client. Called on clean session disconnect. /// Removes all consumers for a client. Called on clean session disconnect.
/// </summary> /// </summary>
/// <param name="clientId">MQTT client identifier.</param>
public void RemoveAllConsumers(string clientId) public void RemoveAllConsumers(string clientId)
{ {
var prefix = $"{clientId}:"; var prefix = $"{clientId}:";
@@ -93,6 +106,9 @@ public sealed class MqttConsumerManager
/// <summary> /// <summary>
/// Gets the binding for a subscription, or null if none exists. /// Gets the binding for a subscription, or null if none exists.
/// </summary> /// </summary>
/// <param name="clientId">MQTT client identifier.</param>
/// <param name="natsSubject">NATS subject mapped from the MQTT topic filter.</param>
/// <returns>Consumer binding for the subscription, or <see langword="null"/>.</returns>
public MqttConsumerBinding? GetBinding(string clientId, string natsSubject) public MqttConsumerBinding? GetBinding(string clientId, string natsSubject)
{ {
return _bindings.TryGetValue($"{clientId}:{natsSubject}", out var binding) ? binding : null; return _bindings.TryGetValue($"{clientId}:{natsSubject}", out var binding) ? binding : null;
@@ -101,6 +117,8 @@ public sealed class MqttConsumerManager
/// <summary> /// <summary>
/// Gets all bindings for a client (for session persistence). /// Gets all bindings for a client (for session persistence).
/// </summary> /// </summary>
/// <param name="clientId">MQTT client identifier.</param>
/// <returns>Per-subscription bindings keyed by NATS subject.</returns>
public IReadOnlyDictionary<string, MqttConsumerBinding> GetClientBindings(string clientId) public IReadOnlyDictionary<string, MqttConsumerBinding> GetClientBindings(string clientId)
{ {
var prefix = $"{clientId}:"; var prefix = $"{clientId}:";
@@ -113,6 +131,9 @@ public sealed class MqttConsumerManager
/// Publishes a message to the $MQTT_msgs stream for QoS delivery. /// Publishes a message to the $MQTT_msgs stream for QoS delivery.
/// Returns the sequence number, or 0 if publish failed. /// Returns the sequence number, or 0 if publish failed.
/// </summary> /// </summary>
/// <param name="natsSubject">NATS subject mapped from MQTT topic.</param>
/// <param name="payload">Message payload bytes.</param>
/// <returns>Stored sequence number, or <c>0</c> if publish failed.</returns>
public ulong PublishToStream(string natsSubject, ReadOnlyMemory<byte> payload) public ulong PublishToStream(string natsSubject, ReadOnlyMemory<byte> payload)
{ {
var subject = $"{MqttProtocolConstants.StreamSubjectPrefix}{natsSubject}"; var subject = $"{MqttProtocolConstants.StreamSubjectPrefix}{natsSubject}";
@@ -129,6 +150,8 @@ public sealed class MqttConsumerManager
/// Acknowledges a message in the stream by removing it (for interest-based retention). /// Acknowledges a message in the stream by removing it (for interest-based retention).
/// Called when PUBACK is received for QoS 1. /// Called when PUBACK is received for QoS 1.
/// </summary> /// </summary>
/// <param name="sequence">Stream sequence to acknowledge and remove.</param>
/// <returns><see langword="true"/> when the sequence was removed.</returns>
public bool AcknowledgeMessage(ulong sequence) public bool AcknowledgeMessage(ulong sequence)
{ {
if (_streamManager.TryGet(MqttProtocolConstants.StreamName, out var handle)) if (_streamManager.TryGet(MqttProtocolConstants.StreamName, out var handle))
@@ -142,6 +165,9 @@ public sealed class MqttConsumerManager
/// <summary> /// <summary>
/// Loads a message from the $MQTT_msgs stream by sequence. /// Loads a message from the $MQTT_msgs stream by sequence.
/// </summary> /// </summary>
/// <param name="sequence">Sequence to load.</param>
/// <param name="ct">Cancellation token.</param>
/// <returns>Stored message, or <see langword="null"/> if not found.</returns>
public async ValueTask<StoredMessage?> LoadMessageAsync(ulong sequence, CancellationToken ct = default) public async ValueTask<StoredMessage?> LoadMessageAsync(ulong sequence, CancellationToken ct = default)
{ {
if (_streamManager.TryGet(MqttProtocolConstants.StreamName, out var handle)) if (_streamManager.TryGet(MqttProtocolConstants.StreamName, out var handle))
@@ -156,6 +182,10 @@ public sealed class MqttConsumerManager
/// Stores a QoS 2 incoming message for deduplication. /// Stores a QoS 2 incoming message for deduplication.
/// Returns the sequence number, or 0 if failed. /// Returns the sequence number, or 0 if failed.
/// </summary> /// </summary>
/// <param name="clientId">MQTT client identifier.</param>
/// <param name="packetId">MQTT packet identifier for QoS 2 flow.</param>
/// <param name="payload">Incoming payload bytes.</param>
/// <returns>Stored sequence number, or <c>0</c> if store failed.</returns>
public ulong StoreQoS2Incoming(string clientId, ushort packetId, ReadOnlyMemory<byte> payload) public ulong StoreQoS2Incoming(string clientId, ushort packetId, ReadOnlyMemory<byte> payload)
{ {
var subject = $"{MqttProtocolConstants.QoS2IncomingMsgsStreamSubjectPrefix}{clientId}.{packetId}"; var subject = $"{MqttProtocolConstants.QoS2IncomingMsgsStreamSubjectPrefix}{clientId}.{packetId}";
@@ -170,6 +200,10 @@ public sealed class MqttConsumerManager
/// <summary> /// <summary>
/// Loads a QoS 2 incoming message for delivery on PUBREL. /// Loads a QoS 2 incoming message for delivery on PUBREL.
/// </summary> /// </summary>
/// <param name="clientId">MQTT client identifier.</param>
/// <param name="packetId">MQTT packet identifier for QoS 2 flow.</param>
/// <param name="ct">Cancellation token.</param>
/// <returns>Stored QoS 2 message, or <see langword="null"/> when missing.</returns>
public async ValueTask<StoredMessage?> LoadQoS2IncomingAsync(string clientId, ushort packetId, CancellationToken ct = default) public async ValueTask<StoredMessage?> LoadQoS2IncomingAsync(string clientId, ushort packetId, CancellationToken ct = default)
{ {
var subject = $"{MqttProtocolConstants.QoS2IncomingMsgsStreamSubjectPrefix}{clientId}.{packetId}"; var subject = $"{MqttProtocolConstants.QoS2IncomingMsgsStreamSubjectPrefix}{clientId}.{packetId}";
@@ -184,6 +218,10 @@ public sealed class MqttConsumerManager
/// <summary> /// <summary>
/// Removes a QoS 2 incoming message after PUBCOMP. /// Removes a QoS 2 incoming message after PUBCOMP.
/// </summary> /// </summary>
/// <param name="clientId">MQTT client identifier.</param>
/// <param name="packetId">MQTT packet identifier for QoS 2 flow.</param>
/// <param name="ct">Cancellation token.</param>
/// <returns><see langword="true"/> when a stored QoS 2 message was removed.</returns>
public async ValueTask<bool> RemoveQoS2IncomingAsync(string clientId, ushort packetId, CancellationToken ct = default) public async ValueTask<bool> RemoveQoS2IncomingAsync(string clientId, ushort packetId, CancellationToken ct = default)
{ {
var subject = $"{MqttProtocolConstants.QoS2IncomingMsgsStreamSubjectPrefix}{clientId}.{packetId}"; var subject = $"{MqttProtocolConstants.QoS2IncomingMsgsStreamSubjectPrefix}{clientId}.{packetId}";
@@ -12,6 +12,10 @@ public sealed class MqttFlowController : IDisposable
private readonly ConcurrentDictionary<string, SubscriptionFlowState> _subscriptions = new(StringComparer.Ordinal); private readonly ConcurrentDictionary<string, SubscriptionFlowState> _subscriptions = new(StringComparer.Ordinal);
private int _defaultMaxAckPending; private int _defaultMaxAckPending;
/// <summary>
/// Initializes MQTT flow control with the default per-subscription outstanding ack limit.
/// </summary>
/// <param name="defaultMaxAckPending">Default max number of in-flight QoS 1/2 publishes per subscription.</param>
public MqttFlowController(int defaultMaxAckPending = 1024) public MqttFlowController(int defaultMaxAckPending = 1024)
{ {
_defaultMaxAckPending = defaultMaxAckPending; _defaultMaxAckPending = defaultMaxAckPending;
@@ -24,6 +28,8 @@ public sealed class MqttFlowController : IDisposable
/// Tries to acquire a slot for sending a QoS message on the given subscription. /// Tries to acquire a slot for sending a QoS message on the given subscription.
/// Returns true if a slot was acquired, false if the limit would be exceeded. /// Returns true if a slot was acquired, false if the limit would be exceeded.
/// </summary> /// </summary>
/// <param name="subscriptionId">Subscription identifier used for per-subscription flow tracking.</param>
/// <param name="ct">Cancellation token for the semaphore wait operation.</param>
public async ValueTask<bool> TryAcquireAsync(string subscriptionId, CancellationToken ct = default) public async ValueTask<bool> TryAcquireAsync(string subscriptionId, CancellationToken ct = default)
{ {
var state = GetOrCreate(subscriptionId); var state = GetOrCreate(subscriptionId);
@@ -33,6 +39,8 @@ public sealed class MqttFlowController : IDisposable
/// <summary> /// <summary>
/// Waits for a slot to become available. Blocks until one is released or cancelled. /// Waits for a slot to become available. Blocks until one is released or cancelled.
/// </summary> /// </summary>
/// <param name="subscriptionId">Subscription identifier used for per-subscription flow tracking.</param>
/// <param name="ct">Cancellation token for the semaphore wait operation.</param>
public async ValueTask AcquireAsync(string subscriptionId, CancellationToken ct = default) public async ValueTask AcquireAsync(string subscriptionId, CancellationToken ct = default)
{ {
var state = GetOrCreate(subscriptionId); var state = GetOrCreate(subscriptionId);
@@ -43,6 +51,7 @@ public sealed class MqttFlowController : IDisposable
/// Releases a slot after receiving PUBACK/PUBCOMP. /// Releases a slot after receiving PUBACK/PUBCOMP.
/// If the semaphore is already at max (duplicate or spurious ack), the release is a no-op. /// If the semaphore is already at max (duplicate or spurious ack), the release is a no-op.
/// </summary> /// </summary>
/// <param name="subscriptionId">Subscription whose pending count should be decremented.</param>
public void Release(string subscriptionId) public void Release(string subscriptionId)
{ {
if (_subscriptions.TryGetValue(subscriptionId, out var state)) if (_subscriptions.TryGetValue(subscriptionId, out var state))
@@ -57,6 +66,7 @@ public sealed class MqttFlowController : IDisposable
/// <summary> /// <summary>
/// Returns the current pending count for a subscription. /// Returns the current pending count for a subscription.
/// </summary> /// </summary>
/// <param name="subscriptionId">Subscription identifier to inspect.</param>
public int GetPendingCount(string subscriptionId) public int GetPendingCount(string subscriptionId)
{ {
if (!_subscriptions.TryGetValue(subscriptionId, out var state)) if (!_subscriptions.TryGetValue(subscriptionId, out var state))
@@ -67,6 +77,7 @@ public sealed class MqttFlowController : IDisposable
/// <summary> /// <summary>
/// Updates the MaxAckPending limit (e.g., on config reload). /// Updates the MaxAckPending limit (e.g., on config reload).
/// </summary> /// </summary>
/// <param name="newLimit">New default in-flight limit for subscriptions created after the update.</param>
public void UpdateLimit(int newLimit) public void UpdateLimit(int newLimit)
{ {
_defaultMaxAckPending = newLimit; _defaultMaxAckPending = newLimit;
@@ -77,6 +88,7 @@ public sealed class MqttFlowController : IDisposable
/// Used to pause JetStream consumer delivery when the limit is reached. /// Used to pause JetStream consumer delivery when the limit is reached.
/// Go reference: server/mqtt.go mqttMaxAckPending flow control. /// Go reference: server/mqtt.go mqttMaxAckPending flow control.
/// </summary> /// </summary>
/// <param name="subscriptionId">Subscription identifier to evaluate.</param>
public bool IsAtCapacity(string subscriptionId) public bool IsAtCapacity(string subscriptionId)
{ {
if (!_subscriptions.TryGetValue(subscriptionId, out var state)) if (!_subscriptions.TryGetValue(subscriptionId, out var state))
@@ -87,6 +99,7 @@ public sealed class MqttFlowController : IDisposable
/// <summary> /// <summary>
/// Removes tracking for a subscription. /// Removes tracking for a subscription.
/// </summary> /// </summary>
/// <param name="subscriptionId">Subscription identifier to remove from flow-control tracking.</param>
public void RemoveSubscription(string subscriptionId) public void RemoveSubscription(string subscriptionId)
{ {
if (_subscriptions.TryRemove(subscriptionId, out var state)) if (_subscriptions.TryRemove(subscriptionId, out var state))
@@ -96,6 +109,9 @@ public sealed class MqttFlowController : IDisposable
/// <summary>Number of tracked subscriptions.</summary> /// <summary>Number of tracked subscriptions.</summary>
public int SubscriptionCount => _subscriptions.Count; public int SubscriptionCount => _subscriptions.Count;
/// <summary>
/// Disposes all semaphore resources tracked for MQTT subscriptions.
/// </summary>
public void Dispose() public void Dispose()
{ {
foreach (var kvp in _subscriptions) foreach (var kvp in _subscriptions)
@@ -114,7 +130,9 @@ public sealed class MqttFlowController : IDisposable
private sealed class SubscriptionFlowState private sealed class SubscriptionFlowState
{ {
/// <summary>Configured maximum pending QoS acknowledgements for this subscription.</summary>
public int MaxAckPending { get; init; } public int MaxAckPending { get; init; }
/// <summary>Semaphore that enforces pending message capacity for this subscription.</summary>
public required SemaphoreSlim Semaphore { get; init; } public required SemaphoreSlim Semaphore { get; init; }
} }
} }
@@ -18,14 +18,25 @@ public sealed class MqttNatsClientAdapter : INatsClient
private readonly MqttConnection _connection; private readonly MqttConnection _connection;
private readonly Dictionary<string, Subscription> _subs = new(StringComparer.Ordinal); private readonly Dictionary<string, Subscription> _subs = new(StringComparer.Ordinal);
/// <summary>Server-assigned client identifier for routing/monitoring.</summary>
public ulong Id { get; } public ulong Id { get; }
/// <summary>Client kind exposed to the NATS routing layer.</summary>
public ClientKind Kind => ClientKind.Client; public ClientKind Kind => ClientKind.Client;
/// <summary>Account currently associated with this MQTT client.</summary>
public Account? Account { get; set; } public Account? Account { get; set; }
/// <summary>CONNECT options are not exposed for MQTT adapter clients.</summary>
public ClientOptions? ClientOpts => null; public ClientOptions? ClientOpts => null;
/// <summary>Resolved permissions for this adapter client.</summary>
public ClientPermissions? Permissions { get; set; } public ClientPermissions? Permissions { get; set; }
/// <summary>MQTT client identifier from the underlying connection.</summary>
public string MqttClientId => _connection.ClientId; public string MqttClientId => _connection.ClientId;
/// <summary>
/// Creates an adapter that exposes an MQTT connection as an <see cref="INatsClient"/>.
/// </summary>
/// <param name="connection">Underlying MQTT connection.</param>
/// <param name="id">Server-assigned adapter/client id.</param>
public MqttNatsClientAdapter(MqttConnection connection, ulong id) public MqttNatsClientAdapter(MqttConnection connection, ulong id)
{ {
_connection = connection; _connection = connection;
@@ -36,6 +47,11 @@ public sealed class MqttNatsClientAdapter : INatsClient
/// Delivers a NATS message to this MQTT client by translating the NATS subject /// Delivers a NATS message to this MQTT client by translating the NATS subject
/// to an MQTT topic and enqueueing a PUBLISH packet into the direct buffer. /// to an MQTT topic and enqueueing a PUBLISH packet into the direct buffer.
/// </summary> /// </summary>
/// <param name="subject">NATS subject being delivered.</param>
/// <param name="sid">Subscription id on this client.</param>
/// <param name="replyTo">Optional reply subject.</param>
/// <param name="headers">Encoded NATS headers.</param>
/// <param name="payload">Message payload bytes.</param>
public void SendMessage(string subject, string sid, string? replyTo, public void SendMessage(string subject, string sid, string? replyTo,
ReadOnlyMemory<byte> headers, ReadOnlyMemory<byte> payload) ReadOnlyMemory<byte> headers, ReadOnlyMemory<byte> payload)
{ {
@@ -47,6 +63,11 @@ public sealed class MqttNatsClientAdapter : INatsClient
/// Enqueues an MQTT PUBLISH into the connection's direct buffer without flushing. /// Enqueues an MQTT PUBLISH into the connection's direct buffer without flushing.
/// Uses cached topic bytes to avoid re-encoding. Zero allocation on the hot path. /// Uses cached topic bytes to avoid re-encoding. Zero allocation on the hot path.
/// </summary> /// </summary>
/// <param name="subject">NATS subject being delivered.</param>
/// <param name="sid">Subscription id on this client.</param>
/// <param name="replyTo">Optional reply subject.</param>
/// <param name="headers">Encoded NATS headers.</param>
/// <param name="payload">Message payload bytes.</param>
public void SendMessageNoFlush(string subject, string sid, string? replyTo, public void SendMessageNoFlush(string subject, string sid, string? replyTo,
ReadOnlyMemory<byte> headers, ReadOnlyMemory<byte> payload) ReadOnlyMemory<byte> headers, ReadOnlyMemory<byte> payload)
{ {
@@ -62,12 +83,21 @@ public sealed class MqttNatsClientAdapter : INatsClient
_connection.SignalMqttFlush(); _connection.SignalMqttFlush();
} }
/// <summary>
/// Queues raw outbound bytes. No-op for MQTT adapter clients.
/// </summary>
/// <param name="data">Raw protocol bytes.</param>
/// <returns>Always <see langword="true"/>.</returns>
public bool QueueOutbound(ReadOnlyMemory<byte> data) public bool QueueOutbound(ReadOnlyMemory<byte> data)
{ {
// No-op for MQTT — binary framing, not raw NATS protocol bytes // No-op for MQTT — binary framing, not raw NATS protocol bytes
return true; return true;
} }
/// <summary>
/// Removes a subscription by id and unregisters it from the account sublist.
/// </summary>
/// <param name="sid">Subscription id to remove.</param>
public void RemoveSubscription(string sid) public void RemoveSubscription(string sid)
{ {
if (_subs.Remove(sid, out var sub)) if (_subs.Remove(sid, out var sub))
@@ -81,6 +111,10 @@ public sealed class MqttNatsClientAdapter : INatsClient
/// Creates a NATS subscription for an MQTT topic filter and inserts it into /// Creates a NATS subscription for an MQTT topic filter and inserts it into
/// the account's SubList so NATS messages are delivered to this MQTT client. /// the account's SubList so NATS messages are delivered to this MQTT client.
/// </summary> /// </summary>
/// <param name="natsSubject">Mapped NATS subject to subscribe to.</param>
/// <param name="sid">Subscription id for tracking/removal.</param>
/// <param name="queue">Optional queue group.</param>
/// <returns>The created subscription instance.</returns>
public Subscription AddSubscription(string natsSubject, string sid, string? queue = null) public Subscription AddSubscription(string natsSubject, string sid, string? queue = null)
{ {
// Pre-warm topic bytes cache for this subject to avoid cache miss on first message. // Pre-warm topic bytes cache for this subject to avoid cache miss on first message.
@@ -114,5 +148,6 @@ public sealed class MqttNatsClientAdapter : INatsClient
_subs.Clear(); _subs.Clear();
} }
/// <summary>Current subscriptions keyed by subscription id.</summary>
public IReadOnlyDictionary<string, Subscription> Subscriptions => _subs; public IReadOnlyDictionary<string, Subscription> Subscriptions => _subs;
} }
+25
View File
@@ -6,8 +6,11 @@ namespace NATS.Server.Mqtt;
/// </summary> /// </summary>
public sealed class MqttJsa public sealed class MqttJsa
{ {
/// <summary>Account that owns the MQTT JetStream operations.</summary>
public string AccountName { get; set; } = string.Empty; public string AccountName { get; set; } = string.Empty;
/// <summary>Reply subject prefix used for MQTT JetStream API calls.</summary>
public string ReplyPrefix { get; set; } = string.Empty; public string ReplyPrefix { get; set; } = string.Empty;
/// <summary>Optional JetStream domain for cross-domain routing.</summary>
public string? Domain { get; set; } public string? Domain { get; set; }
} }
@@ -17,8 +20,11 @@ public sealed class MqttJsa
/// </summary> /// </summary>
public sealed class MqttJsPubMsg public sealed class MqttJsPubMsg
{ {
/// <summary>Target NATS subject for the publish operation.</summary>
public string Subject { get; set; } = string.Empty; public string Subject { get; set; } = string.Empty;
/// <summary>Published payload bytes.</summary>
public byte[] Payload { get; set; } = []; public byte[] Payload { get; set; } = [];
/// <summary>Optional reply subject for request/reply semantics.</summary>
public string? ReplyTo { get; set; } public string? ReplyTo { get; set; }
} }
@@ -28,7 +34,9 @@ public sealed class MqttJsPubMsg
/// </summary> /// </summary>
public sealed class MqttRetMsgDel public sealed class MqttRetMsgDel
{ {
/// <summary>MQTT topic whose retained message should be removed.</summary>
public string Topic { get; set; } = string.Empty; public string Topic { get; set; } = string.Empty;
/// <summary>JetStream sequence of the retained message record.</summary>
public ulong Sequence { get; set; } public ulong Sequence { get; set; }
} }
@@ -38,8 +46,11 @@ public sealed class MqttRetMsgDel
/// </summary> /// </summary>
public sealed class MqttPersistedSession public sealed class MqttPersistedSession
{ {
/// <summary>MQTT client identifier for the persisted session.</summary>
public string ClientId { get; set; } = string.Empty; public string ClientId { get; set; } = string.Empty;
/// <summary>Last issued packet identifier for this session.</summary>
public int LastPacketId { get; set; } public int LastPacketId { get; set; }
/// <summary>Maximum number of unacknowledged QoS deliveries allowed.</summary>
public int MaxAckPending { get; set; } public int MaxAckPending { get; set; }
} }
@@ -49,7 +60,9 @@ public sealed class MqttPersistedSession
/// </summary> /// </summary>
public sealed class MqttRetainedMessageRef public sealed class MqttRetainedMessageRef
{ {
/// <summary>JetStream sequence containing the retained MQTT payload.</summary>
public ulong StreamSequence { get; set; } public ulong StreamSequence { get; set; }
/// <summary>NATS subject mapped from the retained MQTT topic.</summary>
public string Subject { get; set; } = string.Empty; public string Subject { get; set; } = string.Empty;
} }
@@ -59,10 +72,15 @@ public sealed class MqttRetainedMessageRef
/// </summary> /// </summary>
public sealed class MqttSub public sealed class MqttSub
{ {
/// <summary>MQTT topic filter for this subscription.</summary>
public string Filter { get; set; } = string.Empty; public string Filter { get; set; } = string.Empty;
/// <summary>Requested MQTT QoS level.</summary>
public byte Qos { get; set; } public byte Qos { get; set; }
/// <summary>Optional JetStream durable consumer name.</summary>
public string? JsDur { get; set; } public string? JsDur { get; set; }
/// <summary>Indicates whether this is a permanent subscription.</summary>
public bool Prm { get; set; } public bool Prm { get; set; }
/// <summary>Reserved flag kept for Go protocol parity.</summary>
public bool Reserved { get; set; } public bool Reserved { get; set; }
} }
@@ -72,8 +90,11 @@ public sealed class MqttSub
/// </summary> /// </summary>
public sealed class MqttFilter public sealed class MqttFilter
{ {
/// <summary>Original MQTT topic filter.</summary>
public string Filter { get; set; } = string.Empty; public string Filter { get; set; } = string.Empty;
/// <summary>QoS level attached to the filter.</summary>
public byte Qos { get; set; } public byte Qos { get; set; }
/// <summary>Parsed token optimization hint used for dispatch lookups.</summary>
public string? TopicToken { get; set; } public string? TopicToken { get; set; }
} }
@@ -83,8 +104,12 @@ public sealed class MqttFilter
/// </summary> /// </summary>
public sealed class MqttParsedPublishNatsHeader public sealed class MqttParsedPublishNatsHeader
{ {
/// <summary>Subject extracted from MQTT publish headers, when present.</summary>
public string? Subject { get; set; } public string? Subject { get; set; }
/// <summary>Mapped subject after account/topic translation.</summary>
public string? Mapped { get; set; } public string? Mapped { get; set; }
/// <summary>Indicates the packet represents a PUBLISH flow.</summary>
public bool IsPublish { get; set; } public bool IsPublish { get; set; }
/// <summary>Indicates the packet represents a PUBREL flow.</summary>
public bool IsPubRel { get; set; } public bool IsPubRel { get; set; }
} }
+37
View File
@@ -53,6 +53,8 @@ public sealed class MqttRetainedStore
/// An empty payload clears the retained message. /// An empty payload clears the retained message.
/// Go reference: server/mqtt.go mqttHandleRetainedMsg. /// Go reference: server/mqtt.go mqttHandleRetainedMsg.
/// </summary> /// </summary>
/// <param name="topic">MQTT topic for the retained message.</param>
/// <param name="payload">Retained payload bytes; empty clears retained state.</param>
public void SetRetained(string topic, ReadOnlyMemory<byte> payload) public void SetRetained(string topic, ReadOnlyMemory<byte> payload)
{ {
if (payload.IsEmpty) if (payload.IsEmpty)
@@ -69,6 +71,8 @@ public sealed class MqttRetainedStore
/// <summary> /// <summary>
/// Gets the retained message payload for a topic, or null if none. /// Gets the retained message payload for a topic, or null if none.
/// </summary> /// </summary>
/// <param name="topic">MQTT topic to query.</param>
/// <returns>Retained payload, or <see langword="null"/> if none exists.</returns>
public ReadOnlyMemory<byte>? GetRetained(string topic) public ReadOnlyMemory<byte>? GetRetained(string topic)
{ {
if (_retained.TryGetValue(topic, out var payload)) if (_retained.TryGetValue(topic, out var payload))
@@ -82,6 +86,8 @@ public sealed class MqttRetainedStore
/// Supports '+' (single-level) and '#' (multi-level) wildcards. /// Supports '+' (single-level) and '#' (multi-level) wildcards.
/// Go reference: server/mqtt.go mqttGetRetainedMessages ~line 1650. /// Go reference: server/mqtt.go mqttGetRetainedMessages ~line 1650.
/// </summary> /// </summary>
/// <param name="filter">MQTT topic filter.</param>
/// <returns>Matching retained messages.</returns>
public IReadOnlyList<MqttRetainedMessage> GetMatchingRetained(string filter) public IReadOnlyList<MqttRetainedMessage> GetMatchingRetained(string filter)
{ {
var results = new List<MqttRetainedMessage>(); var results = new List<MqttRetainedMessage>();
@@ -100,6 +106,9 @@ public sealed class MqttRetainedStore
/// Returns the number of messages delivered. /// Returns the number of messages delivered.
/// Go reference: server/mqtt.go mqttGetRetainedMessages / mqttHandleRetainedMsg ~line 1650. /// Go reference: server/mqtt.go mqttGetRetainedMessages / mqttHandleRetainedMsg ~line 1650.
/// </summary> /// </summary>
/// <param name="topicFilter">MQTT topic filter for retained delivery.</param>
/// <param name="deliver">Callback invoked for each matching retained message.</param>
/// <returns>Number of retained messages delivered.</returns>
public int DeliverRetainedOnSubscribe(string topicFilter, Action<string, byte[], byte, bool> deliver) public int DeliverRetainedOnSubscribe(string topicFilter, Action<string, byte[], byte, bool> deliver)
{ {
var matches = GetMatchingRetained(topicFilter); var matches = GetMatchingRetained(topicFilter);
@@ -114,6 +123,9 @@ public sealed class MqttRetainedStore
/// Empty payload = tombstone (delete retained). /// Empty payload = tombstone (delete retained).
/// Go reference: server/mqtt.go mqttHandleRetainedMsg with JetStream. /// Go reference: server/mqtt.go mqttHandleRetainedMsg with JetStream.
/// </summary> /// </summary>
/// <param name="topic">MQTT topic for the retained message.</param>
/// <param name="payload">Retained payload bytes; empty clears retained state.</param>
/// <param name="ct">Cancellation token.</param>
public async Task SetRetainedAsync(string topic, ReadOnlyMemory<byte> payload, CancellationToken ct = default) public async Task SetRetainedAsync(string topic, ReadOnlyMemory<byte> payload, CancellationToken ct = default)
{ {
SetRetained(topic, payload); SetRetained(topic, payload);
@@ -138,6 +150,9 @@ public sealed class MqttRetainedStore
/// Gets the retained message, checking backing store if not in memory. /// Gets the retained message, checking backing store if not in memory.
/// Returns null if the topic was explicitly cleared in this session. /// Returns null if the topic was explicitly cleared in this session.
/// </summary> /// </summary>
/// <param name="topic">MQTT topic to query.</param>
/// <param name="ct">Cancellation token.</param>
/// <returns>Retained payload bytes, or <see langword="null"/> if none exists.</returns>
public async Task<byte[]?> GetRetainedAsync(string topic, CancellationToken ct = default) public async Task<byte[]?> GetRetainedAsync(string topic, CancellationToken ct = default)
{ {
var mem = GetRetained(topic); var mem = GetRetained(topic);
@@ -163,6 +178,9 @@ public sealed class MqttRetainedStore
/// Matches an MQTT topic against a filter pattern. /// Matches an MQTT topic against a filter pattern.
/// '+' matches exactly one level, '#' matches zero or more levels (must be last). /// '+' matches exactly one level, '#' matches zero or more levels (must be last).
/// </summary> /// </summary>
/// <param name="topic">Concrete MQTT topic.</param>
/// <param name="filter">MQTT filter pattern.</param>
/// <returns><see langword="true"/> when the topic matches the filter.</returns>
internal static bool MqttTopicMatch(string topic, string filter) internal static bool MqttTopicMatch(string topic, string filter)
{ {
var topicLevels = topic.Split('/'); var topicLevels = topic.Split('/');
@@ -209,7 +227,9 @@ public enum MqttQos2State
/// </summary> /// </summary>
internal sealed class MqttQos2Flow internal sealed class MqttQos2Flow
{ {
/// <summary>Current QoS 2 handshake state for the packet.</summary>
public MqttQos2State State { get; set; } public MqttQos2State State { get; set; }
/// <summary>UTC timestamp when the flow was created.</summary>
public DateTime StartedAtUtc { get; init; } public DateTime StartedAtUtc { get; init; }
} }
@@ -239,6 +259,8 @@ public sealed class MqttQos2StateMachine
/// Begins a new QoS 2 flow for the given packet ID. /// Begins a new QoS 2 flow for the given packet ID.
/// Returns false if a flow for this packet ID already exists (duplicate publish). /// Returns false if a flow for this packet ID already exists (duplicate publish).
/// </summary> /// </summary>
/// <param name="packetId">MQTT packet identifier for the flow.</param>
/// <returns><see langword="true"/> when a new flow was created.</returns>
public bool BeginPublish(ushort packetId) public bool BeginPublish(ushort packetId)
{ {
var flow = new MqttQos2Flow var flow = new MqttQos2Flow
@@ -254,6 +276,8 @@ public sealed class MqttQos2StateMachine
/// Processes a PUBREC for the given packet ID. /// Processes a PUBREC for the given packet ID.
/// Returns false if the flow is not in the expected state. /// Returns false if the flow is not in the expected state.
/// </summary> /// </summary>
/// <param name="packetId">MQTT packet identifier.</param>
/// <returns><see langword="true"/> when the state transition succeeded.</returns>
public bool ProcessPubRec(ushort packetId) public bool ProcessPubRec(ushort packetId)
{ {
if (!_flows.TryGetValue(packetId, out var flow)) if (!_flows.TryGetValue(packetId, out var flow))
@@ -270,6 +294,8 @@ public sealed class MqttQos2StateMachine
/// Processes a PUBREL for the given packet ID. /// Processes a PUBREL for the given packet ID.
/// Returns false if the flow is not in the expected state. /// Returns false if the flow is not in the expected state.
/// </summary> /// </summary>
/// <param name="packetId">MQTT packet identifier.</param>
/// <returns><see langword="true"/> when the state transition succeeded.</returns>
public bool ProcessPubRel(ushort packetId) public bool ProcessPubRel(ushort packetId)
{ {
if (!_flows.TryGetValue(packetId, out var flow)) if (!_flows.TryGetValue(packetId, out var flow))
@@ -287,6 +313,8 @@ public sealed class MqttQos2StateMachine
/// Returns false if the flow is not in the expected state. /// Returns false if the flow is not in the expected state.
/// Removes the flow on completion. /// Removes the flow on completion.
/// </summary> /// </summary>
/// <param name="packetId">MQTT packet identifier.</param>
/// <returns><see langword="true"/> when the flow completed and was removed.</returns>
public bool ProcessPubComp(ushort packetId) public bool ProcessPubComp(ushort packetId)
{ {
if (!_flows.TryGetValue(packetId, out var flow)) if (!_flows.TryGetValue(packetId, out var flow))
@@ -303,6 +331,8 @@ public sealed class MqttQos2StateMachine
/// <summary> /// <summary>
/// Gets the current state for a packet ID, or null if no flow exists. /// Gets the current state for a packet ID, or null if no flow exists.
/// </summary> /// </summary>
/// <param name="packetId">MQTT packet identifier.</param>
/// <returns>Current state, or <see langword="null"/> if no flow exists.</returns>
public MqttQos2State? GetState(ushort packetId) public MqttQos2State? GetState(ushort packetId)
{ {
if (_flows.TryGetValue(packetId, out var flow)) if (_flows.TryGetValue(packetId, out var flow))
@@ -331,6 +361,7 @@ public sealed class MqttQos2StateMachine
/// <summary> /// <summary>
/// Removes a flow (e.g., after timeout cleanup). /// Removes a flow (e.g., after timeout cleanup).
/// </summary> /// </summary>
/// <param name="packetId">MQTT packet identifier.</param>
public void RemoveFlow(ushort packetId) => public void RemoveFlow(ushort packetId) =>
_flows.TryRemove(packetId, out _); _flows.TryRemove(packetId, out _);
@@ -339,6 +370,8 @@ public sealed class MqttQos2StateMachine
/// Alias for <see cref="ProcessPubRec"/> — transitions AwaitingPubRec → AwaitingPubRel. /// Alias for <see cref="ProcessPubRec"/> — transitions AwaitingPubRec → AwaitingPubRel.
/// Returns false if the flow is not in the expected state. /// Returns false if the flow is not in the expected state.
/// </summary> /// </summary>
/// <param name="packetId">MQTT packet identifier.</param>
/// <returns><see langword="true"/> when the state transition succeeded.</returns>
public bool RegisterPubRec(ushort packetId) => ProcessPubRec(packetId); public bool RegisterPubRec(ushort packetId) => ProcessPubRec(packetId);
/// <summary> /// <summary>
@@ -346,6 +379,8 @@ public sealed class MqttQos2StateMachine
/// Alias for <see cref="ProcessPubRel"/> — transitions AwaitingPubRel → AwaitingPubComp. /// Alias for <see cref="ProcessPubRel"/> — transitions AwaitingPubRel → AwaitingPubComp.
/// Returns false if the flow is not in the expected state. /// Returns false if the flow is not in the expected state.
/// </summary> /// </summary>
/// <param name="packetId">MQTT packet identifier.</param>
/// <returns><see langword="true"/> when the state transition succeeded.</returns>
public bool RegisterPubRel(ushort packetId) => ProcessPubRel(packetId); public bool RegisterPubRel(ushort packetId) => ProcessPubRel(packetId);
/// <summary> /// <summary>
@@ -353,5 +388,7 @@ public sealed class MqttQos2StateMachine
/// Alias for <see cref="ProcessPubComp"/> — transitions AwaitingPubComp → Complete and removes the flow. /// Alias for <see cref="ProcessPubComp"/> — transitions AwaitingPubComp → Complete and removes the flow.
/// Returns false if the flow is not in the expected state. /// Returns false if the flow is not in the expected state.
/// </summary> /// </summary>
/// <param name="packetId">MQTT packet identifier.</param>
/// <returns><see langword="true"/> when the flow completed.</returns>
public bool CompletePubComp(ushort packetId) => ProcessPubComp(packetId); public bool CompletePubComp(ushort packetId) => ProcessPubComp(packetId);
} }
+26
View File
@@ -8,39 +8,65 @@ namespace NATS.Server;
public sealed class MqttOptions public sealed class MqttOptions
{ {
// Network // Network
/// <summary>Host interface for the MQTT listener.</summary>
public string Host { get; set; } = ""; public string Host { get; set; } = "";
/// <summary>Port for the MQTT listener.</summary>
public int Port { get; set; } public int Port { get; set; }
// Auth override (MQTT-specific, separate from global auth) // Auth override (MQTT-specific, separate from global auth)
/// <summary>Default user to apply when MQTT clients connect without credentials.</summary>
public string? NoAuthUser { get; set; } public string? NoAuthUser { get; set; }
/// <summary>Optional username required for MQTT authentication.</summary>
public string? Username { get; set; } public string? Username { get; set; }
/// <summary>Optional password required for MQTT authentication.</summary>
public string? Password { get; set; } public string? Password { get; set; }
/// <summary>Optional bearer token accepted for MQTT authentication.</summary>
public string? Token { get; set; } public string? Token { get; set; }
/// <summary>Authentication timeout in seconds for MQTT CONNECT processing.</summary>
public double AuthTimeout { get; set; } public double AuthTimeout { get; set; }
// TLS // TLS
/// <summary>Path to the server certificate used for MQTT TLS.</summary>
public string? TlsCert { get; set; } public string? TlsCert { get; set; }
/// <summary>Path to the private key used for MQTT TLS.</summary>
public string? TlsKey { get; set; } public string? TlsKey { get; set; }
/// <summary>Path to the CA certificate bundle used to validate peer certificates.</summary>
public string? TlsCaCert { get; set; } public string? TlsCaCert { get; set; }
/// <summary>Enables client certificate verification for MQTT TLS connections.</summary>
public bool TlsVerify { get; set; } public bool TlsVerify { get; set; }
/// <summary>TLS handshake timeout in seconds for MQTT clients.</summary>
public double TlsTimeout { get; set; } = 2.0; public double TlsTimeout { get; set; } = 2.0;
/// <summary>Enables TLS certificate subject mapping to users.</summary>
public bool TlsMap { get; set; } public bool TlsMap { get; set; }
/// <summary>Set of pinned client certificate fingerprints allowed for MQTT connections.</summary>
public HashSet<string>? TlsPinnedCerts { get; set; } public HashSet<string>? TlsPinnedCerts { get; set; }
// JetStream integration // JetStream integration
/// <summary>JetStream domain used by MQTT-backed streams and consumers.</summary>
public string? JsDomain { get; set; } public string? JsDomain { get; set; }
/// <summary>Replica count for MQTT-created JetStream streams.</summary>
public int StreamReplicas { get; set; } public int StreamReplicas { get; set; }
/// <summary>Replica count for MQTT-created JetStream consumers.</summary>
public int ConsumerReplicas { get; set; } public int ConsumerReplicas { get; set; }
/// <summary>Stores MQTT JetStream consumer state in memory when enabled.</summary>
public bool ConsumerMemoryStorage { get; set; } public bool ConsumerMemoryStorage { get; set; }
/// <summary>Idle timeout after which inactive MQTT consumers are cleaned up.</summary>
public TimeSpan ConsumerInactiveThreshold { get; set; } public TimeSpan ConsumerInactiveThreshold { get; set; }
// QoS // QoS
/// <summary>Maximum time to wait for QoS acknowledgements before redelivery.</summary>
public TimeSpan AckWait { get; set; } = TimeSpan.FromSeconds(30); public TimeSpan AckWait { get; set; } = TimeSpan.FromSeconds(30);
/// <summary>Maximum number of outstanding unacknowledged QoS messages per consumer.</summary>
public ushort MaxAckPending { get; set; } public ushort MaxAckPending { get; set; }
/// <summary>Timeout for internal JetStream API requests made by MQTT components.</summary>
public TimeSpan JsApiTimeout { get; set; } = TimeSpan.FromSeconds(5); public TimeSpan JsApiTimeout { get; set; } = TimeSpan.FromSeconds(5);
/// <summary>Enables durable MQTT session persistence across reconnects.</summary>
public bool SessionPersistence { get; set; } = true; public bool SessionPersistence { get; set; } = true;
/// <summary>Time-to-live for persisted MQTT session state.</summary>
public TimeSpan SessionTtl { get; set; } = TimeSpan.FromHours(1); public TimeSpan SessionTtl { get; set; } = TimeSpan.FromHours(1);
/// <summary>Enables sending PUBACK for QoS 1 publishes.</summary>
public bool Qos1PubAck { get; set; } = true; public bool Qos1PubAck { get; set; } = true;
/// <summary>Indicates whether MQTT TLS is configured with both certificate and key.</summary>
public bool HasTls => TlsCert != null && TlsKey != null; public bool HasTls => TlsCert != null && TlsKey != null;
} }
+1 -3
View File
@@ -340,9 +340,7 @@ public sealed class NatsClient : INatsClient, IDisposable
return ClientConnectionType.Nats; return ClientConnectionType.Nats;
} }
/// <summary> /// <inheritdoc />
/// Returns a compact connection identity string for diagnostics.
/// </summary>
public override string ToString() public override string ToString()
{ {
var endpoint = RemoteIp is null ? "unknown" : $"{RemoteIp}:{RemotePort}"; var endpoint = RemoteIp is null ? "unknown" : $"{RemoteIp}:{RemotePort}";
+1 -3
View File
@@ -3283,9 +3283,7 @@ public sealed class NatsServer : IMessageRouter, ISubListAccess, IDisposable
_options.SystemAccount = newOpts.SystemAccount; _options.SystemAccount = newOpts.SystemAccount;
} }
/// <summary> /// <inheritdoc />
/// Returns a compact server identity string for diagnostics.
/// </summary>
public override string ToString() public override string ToString()
=> $"NatsServer(ServerId={ServerId}, Name={ServerName}, Addr={Addr()}, Clients={ClientCount})"; => $"NatsServer(ServerId={ServerId}, Name={ServerName}, Addr={Addr()}, Clients={ClientCount})";
+47
View File
@@ -21,16 +21,30 @@ public enum CommandType
public readonly struct ParsedCommand public readonly struct ParsedCommand
{ {
/// <summary>Parsed command type used by server dispatch.</summary>
public CommandType Type { get; init; } public CommandType Type { get; init; }
/// <summary>Original protocol operation token (for example, <c>PUB</c> or <c>SUB</c>).</summary>
public string? Operation { get; init; } public string? Operation { get; init; }
/// <summary>Command subject when the operation carries one.</summary>
public string? Subject { get; init; } public string? Subject { get; init; }
/// <summary>Optional reply subject used for request-reply flows.</summary>
public string? ReplyTo { get; init; } public string? ReplyTo { get; init; }
/// <summary>Queue group name for queue subscriptions.</summary>
public string? Queue { get; init; } public string? Queue { get; init; }
/// <summary>Subscription identifier supplied by the client.</summary>
public string? Sid { get; init; } public string? Sid { get; init; }
/// <summary>Maximum message count for <c>UNSUB</c>; <c>-1</c> when not specified.</summary>
public int MaxMessages { get; init; } public int MaxMessages { get; init; }
/// <summary>Header byte size for <c>HPUB</c>; <c>-1</c> for non-header payloads.</summary>
public int HeaderSize { get; init; } public int HeaderSize { get; init; }
/// <summary>Payload bytes associated with this command, when applicable.</summary>
public ReadOnlyMemory<byte> Payload { get; init; } public ReadOnlyMemory<byte> Payload { get; init; }
/// <summary>
/// Creates a command without payload fields for control-line only operations.
/// </summary>
/// <param name="type">Command type to emit.</param>
/// <param name="operation">Operation token used for tracing and diagnostics.</param>
public static ParsedCommand Simple(CommandType type, string operation) => public static ParsedCommand Simple(CommandType type, string operation) =>
new() { Type = type, Operation = operation, MaxMessages = -1 }; new() { Type = type, Operation = operation, MaxMessages = -1 };
} }
@@ -39,6 +53,7 @@ public sealed class NatsParser
{ {
private static ReadOnlySpan<byte> CrLfBytes => "\r\n"u8; private static ReadOnlySpan<byte> CrLfBytes => "\r\n"u8;
private ILogger? _logger; private ILogger? _logger;
/// <summary>Optional protocol logger used to trace inbound operations.</summary>
public ILogger? Logger { set => _logger = value; } public ILogger? Logger { set => _logger = value; }
// State for split-packet payload reading // State for split-packet payload reading
@@ -50,6 +65,11 @@ public sealed class NatsParser
private CommandType _pendingType; private CommandType _pendingType;
private string _pendingOperation = string.Empty; private string _pendingOperation = string.Empty;
/// <summary>
/// Initializes a parser for the NATS text protocol control and payload frames.
/// </summary>
/// <param name="maxPayload">Reserved for payload policy compatibility; max payload is enforced at connection level.</param>
/// <param name="logger">Optional logger for protocol trace output.</param>
public NatsParser(int maxPayload = NatsProtocol.MaxPayloadSize, ILogger? logger = null) public NatsParser(int maxPayload = NatsProtocol.MaxPayloadSize, ILogger? logger = null)
{ {
_logger = logger; _logger = logger;
@@ -65,6 +85,11 @@ public sealed class NatsParser
_logger.LogTrace("<<- {Op} {Arg}", op, Encoding.ASCII.GetString(arg)); _logger.LogTrace("<<- {Op} {Arg}", op, Encoding.ASCII.GetString(arg));
} }
/// <summary>
/// Attempts to parse one protocol command from the current receive buffer.
/// </summary>
/// <param name="buffer">Receive buffer; advanced past the parsed command on success.</param>
/// <param name="command">Materialized parsed command when parsing succeeds.</param>
public bool TryParse(ref ReadOnlySequence<byte> buffer, out ParsedCommand command) public bool TryParse(ref ReadOnlySequence<byte> buffer, out ParsedCommand command)
{ {
command = default; command = default;
@@ -76,6 +101,11 @@ public sealed class NatsParser
return true; return true;
} }
/// <summary>
/// Attempts to parse one protocol command and return a non-materialized payload view.
/// </summary>
/// <param name="buffer">Receive buffer; advanced past the parsed command on success.</param>
/// <param name="command">Parsed command view that can be materialized by the caller.</param>
internal bool TryParseView(ref ReadOnlySequence<byte> buffer, out ParsedCommandView command) internal bool TryParseView(ref ReadOnlySequence<byte> buffer, out ParsedCommandView command)
{ {
command = default; command = default;
@@ -213,6 +243,12 @@ public sealed class NatsParser
} }
// Go reference: parser.go protoSnippet(start, max, buf). // Go reference: parser.go protoSnippet(start, max, buf).
/// <summary>
/// Builds an ASCII-safe snippet used in protocol violation diagnostics.
/// </summary>
/// <param name="start">Start index in the source buffer.</param>
/// <param name="max">Maximum number of bytes to include in the snippet.</param>
/// <param name="buffer">Source protocol bytes.</param>
internal static string ProtoSnippet(int start, int max, ReadOnlySpan<byte> buffer) internal static string ProtoSnippet(int start, int max, ReadOnlySpan<byte> buffer)
{ {
if (start >= buffer.Length) if (start >= buffer.Length)
@@ -229,6 +265,10 @@ public sealed class NatsParser
return JsonSerializer.Serialize(Encoding.ASCII.GetString(slice)); return JsonSerializer.Serialize(Encoding.ASCII.GetString(slice));
} }
/// <summary>
/// Builds a protocol snippet using the default snippet length.
/// </summary>
/// <param name="buffer">Source protocol bytes.</param>
internal static string ProtoSnippet(ReadOnlySpan<byte> buffer) => internal static string ProtoSnippet(ReadOnlySpan<byte> buffer) =>
ProtoSnippet(0, NatsProtocol.ProtoSnippetSize, buffer); ProtoSnippet(0, NatsProtocol.ProtoSnippetSize, buffer);
@@ -468,6 +508,7 @@ public sealed class NatsParser
/// <summary> /// <summary>
/// Parse a decimal integer from ASCII bytes. Returns -1 on error. /// Parse a decimal integer from ASCII bytes. Returns -1 on error.
/// </summary> /// </summary>
/// <param name="data">ASCII digit span containing a non-negative integer.</param>
internal static int ParseSize(Span<byte> data) internal static int ParseSize(Span<byte> data)
{ {
if (data.Length == 0 || data.Length > 9) if (data.Length == 0 || data.Length > 9)
@@ -487,6 +528,8 @@ public sealed class NatsParser
/// Split by spaces/tabs into argument ranges. Returns the number of arguments found. /// Split by spaces/tabs into argument ranges. Returns the number of arguments found.
/// Uses Span&lt;Range&gt; for zero-allocation argument splitting. /// Uses Span&lt;Range&gt; for zero-allocation argument splitting.
/// </summary> /// </summary>
/// <param name="data">Control-line argument bytes.</param>
/// <param name="ranges">Destination range buffer populated with argument spans.</param>
internal static int SplitArgs(Span<byte> data, Span<Range> ranges) internal static int SplitArgs(Span<byte> data, Span<Range> ranges)
{ {
int count = 0; int count = 0;
@@ -525,6 +568,10 @@ public sealed class NatsParser
public class ProtocolViolationException : Exception public class ProtocolViolationException : Exception
{ {
/// <summary>
/// Initializes an exception for malformed or protocol-incompatible client frames.
/// </summary>
/// <param name="message">Human-readable protocol violation details.</param>
public ProtocolViolationException(string message) public ProtocolViolationException(string message)
: base(message) : base(message)
{ {
+28
View File
@@ -10,13 +10,19 @@ namespace NATS.Server.Protocol;
/// </summary> /// </summary>
public sealed class ProxyAddress public sealed class ProxyAddress
{ {
/// <summary>Source IP address reported by PROXY protocol.</summary>
public required IPAddress SrcIp { get; init; } public required IPAddress SrcIp { get; init; }
/// <summary>Source TCP port reported by PROXY protocol.</summary>
public required ushort SrcPort { get; init; } public required ushort SrcPort { get; init; }
/// <summary>Destination IP address reported by PROXY protocol.</summary>
public required IPAddress DstIp { get; init; } public required IPAddress DstIp { get; init; }
/// <summary>Destination TCP port reported by PROXY protocol.</summary>
public required ushort DstPort { get; init; } public required ushort DstPort { get; init; }
/// <summary>Network family derived from source address (<c>tcp4</c> or <c>tcp6</c>).</summary>
public string Network => SrcIp.AddressFamily == System.Net.Sockets.AddressFamily.InterNetwork ? "tcp4" : "tcp6"; public string Network => SrcIp.AddressFamily == System.Net.Sockets.AddressFamily.InterNetwork ? "tcp4" : "tcp6";
/// <inheritdoc />
public override string ToString() => public override string ToString() =>
SrcIp.AddressFamily == System.Net.Sockets.AddressFamily.InterNetworkV6 SrcIp.AddressFamily == System.Net.Sockets.AddressFamily.InterNetworkV6
? $"[{SrcIp}]:{SrcPort}" ? $"[{SrcIp}]:{SrcPort}"
@@ -36,7 +42,9 @@ public enum ProxyParseResultKind
public sealed class ProxyParseResult public sealed class ProxyParseResult
{ {
/// <summary>Result kind indicating proxy or local passthrough.</summary>
public required ProxyParseResultKind Kind { get; init; } public required ProxyParseResultKind Kind { get; init; }
/// <summary>Parsed address payload when <see cref="Kind"/> is <see cref="ProxyParseResultKind.Proxy"/>.</summary>
public ProxyAddress? Address { get; init; } public ProxyAddress? Address { get; init; }
} }
@@ -88,6 +96,8 @@ public static class ProxyProtocolParser
/// entire header (up to the CRLF for v1, or the full fixed+address block for v2). /// entire header (up to the CRLF for v1, or the full fixed+address block for v2).
/// Throws <see cref="ProxyProtocolException"/> for malformed input. /// Throws <see cref="ProxyProtocolException"/> for malformed input.
/// </summary> /// </summary>
/// <param name="data">Raw bytes containing a complete PROXY header.</param>
/// <returns>Parsed PROXY result including kind and optional address.</returns>
public static ProxyParseResult Parse(ReadOnlySpan<byte> data) public static ProxyParseResult Parse(ReadOnlySpan<byte> data)
{ {
if (data.Length < 6) if (data.Length < 6)
@@ -114,6 +124,8 @@ public static class ProxyProtocolParser
/// Expects the "PROXY " prefix (6 bytes) to have already been stripped. /// Expects the "PROXY " prefix (6 bytes) to have already been stripped.
/// Reference: readProxyProtoV1Header (client_proxyproto.go:134) /// Reference: readProxyProtoV1Header (client_proxyproto.go:134)
/// </summary> /// </summary>
/// <param name="afterPrefix">Bytes following the stripped <c>PROXY </c> prefix.</param>
/// <returns>Parsed PROXY result including kind and optional address.</returns>
public static ProxyParseResult ParseV1(ReadOnlySpan<byte> afterPrefix) public static ProxyParseResult ParseV1(ReadOnlySpan<byte> afterPrefix)
{ {
if (afterPrefix.Length > V1MaxLineLen - 6) if (afterPrefix.Length > V1MaxLineLen - 6)
@@ -187,6 +199,8 @@ public static class ProxyProtocolParser
/// Parses a full PROXY protocol v2 binary header including signature. /// Parses a full PROXY protocol v2 binary header including signature.
/// Reference: readProxyProtoV2Header / parseProxyProtoV2Header (client_proxyproto.go:274) /// Reference: readProxyProtoV2Header / parseProxyProtoV2Header (client_proxyproto.go:274)
/// </summary> /// </summary>
/// <param name="data">Raw bytes containing full PROXY v2 header.</param>
/// <returns>Parsed PROXY result including kind and optional address.</returns>
public static ProxyParseResult ParseV2(ReadOnlySpan<byte> data) public static ProxyParseResult ParseV2(ReadOnlySpan<byte> data)
{ {
if (data.Length < V2HeaderSize) if (data.Length < V2HeaderSize)
@@ -205,6 +219,8 @@ public static class ProxyProtocolParser
/// 12-byte signature, then the variable-length address block. /// 12-byte signature, then the variable-length address block.
/// Reference: parseProxyProtoV2Header (client_proxyproto.go:301) /// Reference: parseProxyProtoV2Header (client_proxyproto.go:301)
/// </summary> /// </summary>
/// <param name="header">Bytes immediately after v2 signature.</param>
/// <returns>Parsed PROXY result including kind and optional address.</returns>
public static ProxyParseResult ParseV2AfterSig(ReadOnlySpan<byte> header) public static ProxyParseResult ParseV2AfterSig(ReadOnlySpan<byte> header)
{ {
if (header.Length < 4) if (header.Length < 4)
@@ -290,6 +306,12 @@ public static class ProxyProtocolParser
// ------------------------------------------------------------------------- // -------------------------------------------------------------------------
/// <summary>Builds a valid PROXY v2 binary header for the given parameters.</summary> /// <summary>Builds a valid PROXY v2 binary header for the given parameters.</summary>
/// <param name="srcIp">Source IP address.</param>
/// <param name="dstIp">Destination IP address.</param>
/// <param name="srcPort">Source TCP port.</param>
/// <param name="dstPort">Destination TCP port.</param>
/// <param name="isIPv6">Set to <see langword="true"/> to emit IPv6 header family.</param>
/// <returns>Encoded PROXY v2 header bytes.</returns>
public static byte[] BuildV2Header( public static byte[] BuildV2Header(
string srcIp, string dstIp, ushort srcPort, ushort dstPort, bool isIPv6 = false) string srcIp, string dstIp, ushort srcPort, ushort dstPort, bool isIPv6 = false)
{ {
@@ -339,6 +361,12 @@ public static class ProxyProtocolParser
} }
/// <summary>Builds a PROXY v1 text header.</summary> /// <summary>Builds a PROXY v1 text header.</summary>
/// <param name="protocol">Protocol token (e.g. <c>TCP4</c>, <c>TCP6</c>, <c>UNKNOWN</c>).</param>
/// <param name="srcIp">Source IP address.</param>
/// <param name="dstIp">Destination IP address.</param>
/// <param name="srcPort">Source TCP port.</param>
/// <param name="dstPort">Destination TCP port.</param>
/// <returns>Encoded PROXY v1 header bytes.</returns>
public static byte[] BuildV1Header( public static byte[] BuildV1Header(
string protocol, string srcIp, string dstIp, ushort srcPort, ushort dstPort) string protocol, string srcIp, string dstIp, ushort srcPort, ushort dstPort)
{ {
+25
View File
@@ -73,6 +73,11 @@ public sealed class NatsRaftTransport : IRaftTransport
/// ///
/// Go: server/raft.go:2854-2916 (sendAppendEntry / sendAppendEntryLocked) /// Go: server/raft.go:2854-2916 (sendAppendEntry / sendAppendEntryLocked)
/// </summary> /// </summary>
/// <param name="leaderId">Leader node id sending append entries.</param>
/// <param name="followerIds">Follower node ids targeted for replication.</param>
/// <param name="entry">Log entry to replicate.</param>
/// <param name="ct">Cancellation token.</param>
/// <returns>Per-follower append dispatch results.</returns>
public Task<IReadOnlyList<AppendResult>> AppendEntriesAsync( public Task<IReadOnlyList<AppendResult>> AppendEntriesAsync(
string leaderId, string leaderId,
IReadOnlyList<string> followerIds, IReadOnlyList<string> followerIds,
@@ -115,6 +120,11 @@ public sealed class NatsRaftTransport : IRaftTransport
/// ///
/// Go: server/raft.go:3594-3630 (requestVote / sendVoteRequest) /// Go: server/raft.go:3594-3630 (requestVote / sendVoteRequest)
/// </summary> /// </summary>
/// <param name="candidateId">Candidate node id requesting votes.</param>
/// <param name="voterId">Target voter node id.</param>
/// <param name="request">Vote request payload.</param>
/// <param name="ct">Cancellation token.</param>
/// <returns>Vote response placeholder for transport dispatch.</returns>
public Task<VoteResponse> RequestVoteAsync( public Task<VoteResponse> RequestVoteAsync(
string candidateId, string candidateId,
string voterId, string voterId,
@@ -149,6 +159,10 @@ public sealed class NatsRaftTransport : IRaftTransport
/// Go: server/raft.go:3247 (buildSnapshotAppendEntry), /// Go: server/raft.go:3247 (buildSnapshotAppendEntry),
/// raft.go:2168 — raftCatchupReply = "$NRG.CR.%s" /// raft.go:2168 — raftCatchupReply = "$NRG.CR.%s"
/// </summary> /// </summary>
/// <param name="leaderId">Leader node id sending snapshot.</param>
/// <param name="followerId">Follower node id receiving snapshot.</param>
/// <param name="snapshot">Snapshot payload to install.</param>
/// <param name="ct">Cancellation token.</param>
public Task InstallSnapshotAsync( public Task InstallSnapshotAsync(
string leaderId, string leaderId,
string followerId, string followerId,
@@ -179,6 +193,7 @@ public sealed class NatsRaftTransport : IRaftTransport
/// ///
/// Go: server/raft.go:949 — ForwardProposal → n.sendq.push to n.psubj /// Go: server/raft.go:949 — ForwardProposal → n.sendq.push to n.psubj
/// </summary> /// </summary>
/// <param name="entry">Serialized proposal bytes.</param>
public void ForwardProposal(ReadOnlyMemory<byte> entry) public void ForwardProposal(ReadOnlyMemory<byte> entry)
{ {
var proposalSubject = RaftSubjects.Proposal(_groupId); var proposalSubject = RaftSubjects.Proposal(_groupId);
@@ -192,6 +207,7 @@ public sealed class NatsRaftTransport : IRaftTransport
/// ///
/// Go: server/raft.go:986 — ProposeRemovePeer → n.sendq.push to n.rpsubj /// Go: server/raft.go:986 — ProposeRemovePeer → n.sendq.push to n.rpsubj
/// </summary> /// </summary>
/// <param name="peer">Peer id to remove from the group.</param>
public void ProposeRemovePeer(string peer) public void ProposeRemovePeer(string peer)
{ {
var removePeerSubject = RaftSubjects.RemovePeer(_groupId); var removePeerSubject = RaftSubjects.RemovePeer(_groupId);
@@ -209,6 +225,10 @@ public sealed class NatsRaftTransport : IRaftTransport
/// ///
/// Go reference: raft.go sendTimeoutNow /// Go reference: raft.go sendTimeoutNow
/// </summary> /// </summary>
/// <param name="leaderId">Leader node id issuing TimeoutNow.</param>
/// <param name="targetId">Target follower node id.</param>
/// <param name="term">Leader term for the request.</param>
/// <param name="ct">Cancellation token.</param>
public Task SendTimeoutNowAsync(string leaderId, string targetId, ulong term, CancellationToken ct) public Task SendTimeoutNowAsync(string leaderId, string targetId, ulong term, CancellationToken ct)
{ {
_ = targetId; _ = targetId;
@@ -226,6 +246,11 @@ public sealed class NatsRaftTransport : IRaftTransport
/// ///
/// Go reference: raft.go — leader sends empty AppendEntries to confirm quorum for reads. /// Go reference: raft.go — leader sends empty AppendEntries to confirm quorum for reads.
/// </summary> /// </summary>
/// <param name="leaderId">Leader node id sending heartbeats.</param>
/// <param name="followerIds">Follower node ids targeted by heartbeats.</param>
/// <param name="term">Leader term used in heartbeat payloads.</param>
/// <param name="onAck">Callback invoked per follower dispatch.</param>
/// <param name="ct">Cancellation token.</param>
public Task SendHeartbeatAsync( public Task SendHeartbeatAsync(
string leaderId, string leaderId,
IReadOnlyList<string> followerIds, IReadOnlyList<string> followerIds,
+213
View File
@@ -55,41 +55,110 @@ public sealed class RaftNode : IDisposable
// Pre-vote: Go NATS server does not implement pre-vote (RFC 5849 §9.6). Skipped for parity. // Pre-vote: Go NATS server does not implement pre-vote (RFC 5849 §9.6). Skipped for parity.
/// <summary>
/// Unique node identifier in this RAFT group.
/// </summary>
public string Id { get; } public string Id { get; }
/// <summary>
/// Logical RAFT group name used for shared consensus state.
/// </summary>
public string GroupName { get; } public string GroupName { get; }
/// <summary>
/// UTC timestamp when this node instance was created.
/// </summary>
public DateTime CreatedUtc => _createdUtc; public DateTime CreatedUtc => _createdUtc;
/// <summary>
/// Current RAFT term tracked by this node.
/// </summary>
public int Term => TermState.CurrentTerm; public int Term => TermState.CurrentTerm;
/// <summary>
/// Indicates whether this node is currently leader.
/// </summary>
public bool IsLeader => Role == RaftRole.Leader; public bool IsLeader => Role == RaftRole.Leader;
/// <summary>
/// UTC time when this node last became leader.
/// </summary>
public DateTime? LeaderSince => _leaderSinceUtc; public DateTime? LeaderSince => _leaderSinceUtc;
/// <summary>
/// Current leader id for this group, or empty when unknown.
/// </summary>
public string GroupLeader => _groupLeader; public string GroupLeader => _groupLeader;
/// <summary>
/// True when no leader is currently known.
/// </summary>
public bool Leaderless => string.IsNullOrEmpty(_groupLeader); public bool Leaderless => string.IsNullOrEmpty(_groupLeader);
/// <summary>
/// True once any leader has previously been observed.
/// </summary>
public bool HadPreviousLeader => _hadPreviousLeader; public bool HadPreviousLeader => _hadPreviousLeader;
/// <summary>
/// Current RAFT role for this node.
/// </summary>
public RaftRole Role { get; private set; } = RaftRole.Follower; public RaftRole Role { get; private set; } = RaftRole.Follower;
/// <summary>
/// Indicates observer mode (non-voting) status.
/// </summary>
public bool IsObserver => _observerMode; public bool IsObserver => _observerMode;
/// <summary>
/// Indicates whether this node has been deleted.
/// </summary>
public bool IsDeleted => _isDeleted; public bool IsDeleted => _isDeleted;
/// <summary>
/// Active member ids in the current configuration.
/// </summary>
public IReadOnlyCollection<string> Members => _members; public IReadOnlyCollection<string> Members => _members;
/// <summary>
/// Durable term and vote state.
/// </summary>
public RaftTermState TermState { get; } = new(); public RaftTermState TermState { get; } = new();
/// <summary>
/// Highest applied log index.
/// </summary>
public long AppliedIndex { get; set; } public long AppliedIndex { get; set; }
/// <summary>
/// In-memory RAFT log for this node.
/// </summary>
public RaftLog Log { get; private set; } = new(); public RaftLog Log { get; private set; } = new();
// B1: Commit tracking // B1: Commit tracking
// Go reference: raft.go:150-160 (applied/processed fields), raft.go:2100-2150 (ApplyQ) // Go reference: raft.go:150-160 (applied/processed fields), raft.go:2100-2150 (ApplyQ)
/// <summary>
/// Highest committed index acknowledged by quorum.
/// </summary>
public long CommitIndex { get; private set; } public long CommitIndex { get; private set; }
/// <summary>
/// Highest index processed by the state machine.
/// </summary>
public long ProcessedIndex { get; private set; } public long ProcessedIndex { get; private set; }
/// <summary>
/// Queue of committed entries awaiting apply.
/// </summary>
public CommitQueue<RaftLogEntry> CommitQueue { get; } = new(); public CommitQueue<RaftLogEntry> CommitQueue { get; } = new();
// B2: Election timeout configuration (milliseconds) // B2: Election timeout configuration (milliseconds)
/// <summary>
/// Minimum election timeout jitter bound in milliseconds.
/// </summary>
public int ElectionTimeoutMinMs { get; set; } = 150; public int ElectionTimeoutMinMs { get; set; } = 150;
/// <summary>
/// Maximum election timeout jitter bound in milliseconds.
/// </summary>
public int ElectionTimeoutMaxMs { get; set; } = 300; public int ElectionTimeoutMaxMs { get; set; } = 300;
// B6: Pre-vote protocol // B6: Pre-vote protocol
// Go reference: raft.go:1600-1700 (pre-vote logic) // Go reference: raft.go:1600-1700 (pre-vote logic)
// When enabled, a node first conducts a pre-vote round before starting a real election. // When enabled, a node first conducts a pre-vote round before starting a real election.
// This prevents partitioned nodes from disrupting the cluster by incrementing terms. // This prevents partitioned nodes from disrupting the cluster by incrementing terms.
/// <summary>
/// Enables pre-vote rounds before real elections.
/// </summary>
public bool PreVoteEnabled { get; set; } = true; public bool PreVoteEnabled { get; set; } = true;
// B4: True while a membership change log entry is pending quorum. // B4: True while a membership change log entry is pending quorum.
// Go reference: raft.go:961-1019 single-change invariant. // Go reference: raft.go:961-1019 single-change invariant.
/// <summary>
/// True while a membership-change entry is pending commit.
/// </summary>
public bool MembershipChangeInProgress => Interlocked.Read(ref _membershipChangeIndex) > 0; public bool MembershipChangeInProgress => Interlocked.Read(ref _membershipChangeIndex) > 0;
/// <summary> /// <summary>
@@ -124,6 +193,15 @@ public sealed class RaftNode : IDisposable
// Go reference: raft.go resetElectionTimeout (uses rand.Int63n for jitter) // Go reference: raft.go resetElectionTimeout (uses rand.Int63n for jitter)
private Random _random; private Random _random;
/// <summary>
/// Creates a RAFT node with optional transport and persistence wiring.
/// </summary>
/// <param name="id">Node id.</param>
/// <param name="transport">Transport for peer RPCs.</param>
/// <param name="persistDirectory">Optional persistence directory for term/log metadata.</param>
/// <param name="compactionOptions">Optional log compaction policy.</param>
/// <param name="random">Optional random source for deterministic timeout tests.</param>
/// <param name="group">Optional RAFT group name.</param>
public RaftNode(string id, IRaftTransport? transport = null, string? persistDirectory = null, public RaftNode(string id, IRaftTransport? transport = null, string? persistDirectory = null,
CompactionOptions? compactionOptions = null, Random? random = null, string? group = null) CompactionOptions? compactionOptions = null, Random? random = null, string? group = null)
{ {
@@ -138,6 +216,10 @@ public sealed class RaftNode : IDisposable
_random = random ?? Random.Shared; _random = random ?? Random.Shared;
} }
/// <summary>
/// Configures known cluster peers and resets membership tracking from that set.
/// </summary>
/// <param name="peers">Peers participating in the same RAFT group.</param>
public void ConfigureCluster(IEnumerable<RaftNode> peers) public void ConfigureCluster(IEnumerable<RaftNode> peers)
{ {
var configuredPeers = peers as ICollection<RaftNode> ?? peers.ToList(); var configuredPeers = peers as ICollection<RaftNode> ?? peers.ToList();
@@ -163,10 +245,22 @@ public sealed class RaftNode : IDisposable
_clusterSize = Math.Max(configuredPeers.Count, 1); _clusterSize = Math.Max(configuredPeers.Count, 1);
} }
/// <summary>
/// Adds a member id to the active configuration.
/// </summary>
/// <param name="memberId">Member id to add.</param>
public void AddMember(string memberId) => _members.Add(memberId); public void AddMember(string memberId) => _members.Add(memberId);
/// <summary>
/// Removes a member id from the active configuration.
/// </summary>
/// <param name="memberId">Member id to remove.</param>
public void RemoveMember(string memberId) => _members.Remove(memberId); public void RemoveMember(string memberId) => _members.Remove(memberId);
/// <summary>
/// Starts a new election term and attempts leadership with self-vote.
/// </summary>
/// <param name="clusterSize">Current cluster size used for quorum math.</param>
public void StartElection(int clusterSize) public void StartElection(int clusterSize)
{ {
_groupLeader = NoLeader; _groupLeader = NoLeader;
@@ -178,6 +272,11 @@ public sealed class RaftNode : IDisposable
TryBecomeLeader(clusterSize); TryBecomeLeader(clusterSize);
} }
/// <summary>
/// Evaluates and records a vote request for the supplied candidate and term.
/// </summary>
/// <param name="term">Candidate term.</param>
/// <param name="candidateId">Candidate id requesting vote.</param>
public VoteResponse GrantVote(int term, string candidateId = "") public VoteResponse GrantVote(int term, string candidateId = "")
{ {
if (term < TermState.CurrentTerm) if (term < TermState.CurrentTerm)
@@ -199,6 +298,11 @@ public sealed class RaftNode : IDisposable
return new VoteResponse { Granted = true }; return new VoteResponse { Granted = true };
} }
/// <summary>
/// Processes leader heartbeat and refreshes follower election timeout state.
/// </summary>
/// <param name="term">Leader term from heartbeat.</param>
/// <param name="fromPeerId">Optional sending leader id.</param>
public void ReceiveHeartbeat(int term, string? fromPeerId = null) public void ReceiveHeartbeat(int term, string? fromPeerId = null)
{ {
if (term < TermState.CurrentTerm) if (term < TermState.CurrentTerm)
@@ -224,6 +328,11 @@ public sealed class RaftNode : IDisposable
} }
} }
/// <summary>
/// Processes a vote response while campaigning and checks leader transition.
/// </summary>
/// <param name="response">Vote response from a peer.</param>
/// <param name="clusterSize">Cluster size used for quorum math.</param>
public void ReceiveVote(VoteResponse response, int clusterSize = 3) public void ReceiveVote(VoteResponse response, int clusterSize = 3)
{ {
if (!response.Granted) if (!response.Granted)
@@ -354,6 +463,11 @@ public sealed class RaftNode : IDisposable
"The leader may be partitioned."); "The leader may be partitioned.");
} }
/// <summary>
/// Proposes a single command and waits for quorum replication.
/// </summary>
/// <param name="command">Command payload to append to the RAFT log.</param>
/// <param name="ct">Cancellation token for replication work.</param>
public async ValueTask<long> ProposeAsync(string command, CancellationToken ct) public async ValueTask<long> ProposeAsync(string command, CancellationToken ct)
{ {
if (Role != RaftRole.Leader) if (Role != RaftRole.Leader)
@@ -415,6 +529,8 @@ public sealed class RaftNode : IDisposable
/// Proposes a batch of commands in order and returns their resulting indexes. /// Proposes a batch of commands in order and returns their resulting indexes.
/// Go reference: raft.go ProposeMulti. /// Go reference: raft.go ProposeMulti.
/// </summary> /// </summary>
/// <param name="commands">Commands to propose sequentially.</param>
/// <param name="ct">Cancellation token for replication work.</param>
public async ValueTask<IReadOnlyList<long>> ProposeMultiAsync(IEnumerable<string> commands, CancellationToken ct) public async ValueTask<IReadOnlyList<long>> ProposeMultiAsync(IEnumerable<string> commands, CancellationToken ct)
{ {
ArgumentNullException.ThrowIfNull(commands); ArgumentNullException.ThrowIfNull(commands);
@@ -429,6 +545,10 @@ public sealed class RaftNode : IDisposable
return indexes; return indexes;
} }
/// <summary>
/// Marks an index applied and returns approximate applied entry and byte counts.
/// </summary>
/// <param name="index">Applied index boundary.</param>
public (long Entries, long Bytes) Applied(long index) public (long Entries, long Bytes) Applied(long index)
{ {
MarkProcessed(index); MarkProcessed(index);
@@ -448,6 +568,8 @@ public sealed class RaftNode : IDisposable
/// After the entry reaches quorum the peer is added to _members. /// After the entry reaches quorum the peer is added to _members.
/// Go reference: raft.go:961-990 (proposeAddPeer). /// Go reference: raft.go:961-990 (proposeAddPeer).
/// </summary> /// </summary>
/// <param name="peerId">Peer id to add.</param>
/// <param name="ct">Cancellation token for replication work.</param>
public async ValueTask<long> ProposeAddPeerAsync(string peerId, CancellationToken ct) public async ValueTask<long> ProposeAddPeerAsync(string peerId, CancellationToken ct)
{ {
if (Role != RaftRole.Leader) if (Role != RaftRole.Leader)
@@ -496,6 +618,8 @@ public sealed class RaftNode : IDisposable
/// Only the leader may propose; only one membership change may be in flight at a time. /// Only the leader may propose; only one membership change may be in flight at a time.
/// Go reference: raft.go:992-1019 (proposeRemovePeer). /// Go reference: raft.go:992-1019 (proposeRemovePeer).
/// </summary> /// </summary>
/// <param name="peerId">Peer id to remove.</param>
/// <param name="ct">Cancellation token for replication work.</param>
public async ValueTask<long> ProposeRemovePeerAsync(string peerId, CancellationToken ct) public async ValueTask<long> ProposeRemovePeerAsync(string peerId, CancellationToken ct)
{ {
if (Role != RaftRole.Leader) if (Role != RaftRole.Leader)
@@ -545,6 +669,8 @@ public sealed class RaftNode : IDisposable
/// are replicated to all nodes that participate in either configuration. /// are replicated to all nodes that participate in either configuration.
/// Go reference: raft.go Section 4 (joint consensus). /// Go reference: raft.go Section 4 (joint consensus).
/// </summary> /// </summary>
/// <param name="cold">Old voter configuration.</param>
/// <param name="cnew">New voter configuration.</param>
public void BeginJointConsensus(IReadOnlyCollection<string> cold, IReadOnlyCollection<string> cnew) public void BeginJointConsensus(IReadOnlyCollection<string> cold, IReadOnlyCollection<string> cnew)
{ {
_jointOldMembers = new HashSet<string>(cold, StringComparer.Ordinal); _jointOldMembers = new HashSet<string>(cold, StringComparer.Ordinal);
@@ -579,6 +705,8 @@ public sealed class RaftNode : IDisposable
/// Returns false when not in joint consensus. /// Returns false when not in joint consensus.
/// Go reference: raft.go Section 4 — joint config quorum calculation. /// Go reference: raft.go Section 4 — joint config quorum calculation.
/// </summary> /// </summary>
/// <param name="coldVoters">Acknowledging voters from old configuration.</param>
/// <param name="cnewVoters">Acknowledging voters from new configuration.</param>
public bool CalculateJointQuorum( public bool CalculateJointQuorum(
IReadOnlyCollection<string> coldVoters, IReadOnlyCollection<string> coldVoters,
IReadOnlyCollection<string> cnewVoters) IReadOnlyCollection<string> cnewVoters)
@@ -600,6 +728,7 @@ public sealed class RaftNode : IDisposable
/// do not need to be replayed on restart. /// do not need to be replayed on restart.
/// Go reference: raft.go CreateSnapshotCheckpoint. /// Go reference: raft.go CreateSnapshotCheckpoint.
/// </summary> /// </summary>
/// <param name="ct">Cancellation token for snapshot persistence.</param>
public async Task<RaftSnapshot> CreateSnapshotCheckpointAsync(CancellationToken ct) public async Task<RaftSnapshot> CreateSnapshotCheckpointAsync(CancellationToken ct)
{ {
var snapshot = new RaftSnapshot var snapshot = new RaftSnapshot
@@ -618,6 +747,8 @@ public sealed class RaftNode : IDisposable
/// apply pipeline, discards pending entries, then fast-forwards to the snapshot state. /// apply pipeline, discards pending entries, then fast-forwards to the snapshot state.
/// Go reference: raft.go DrainAndReplaySnapshot. /// Go reference: raft.go DrainAndReplaySnapshot.
/// </summary> /// </summary>
/// <param name="snapshot">Snapshot to install.</param>
/// <param name="ct">Cancellation token for snapshot persistence.</param>
public async Task DrainAndReplaySnapshotAsync(RaftSnapshot snapshot, CancellationToken ct) public async Task DrainAndReplaySnapshotAsync(RaftSnapshot snapshot, CancellationToken ct)
{ {
// Drain any pending commit-queue entries that are now superseded by the snapshot // Drain any pending commit-queue entries that are now superseded by the snapshot
@@ -745,17 +876,27 @@ public sealed class RaftNode : IDisposable
/// Marks the given index as processed by the state machine. /// Marks the given index as processed by the state machine.
/// Go reference: raft.go applied/processed tracking. /// Go reference: raft.go applied/processed tracking.
/// </summary> /// </summary>
/// <param name="index">Processed index boundary.</param>
public void MarkProcessed(long index) public void MarkProcessed(long index)
{ {
if (index > ProcessedIndex) if (index > ProcessedIndex)
ProcessedIndex = index; ProcessedIndex = index;
} }
/// <summary>
/// Appends a replicated entry received from leader into local log state.
/// </summary>
/// <param name="entry">Replicated log entry.</param>
public void ReceiveReplicatedEntry(RaftLogEntry entry) public void ReceiveReplicatedEntry(RaftLogEntry entry)
{ {
Log.AppendReplicated(entry); Log.AppendReplicated(entry);
} }
/// <summary>
/// Validates term and appends a leader entry while refreshing election timeout.
/// </summary>
/// <param name="entry">Replicated entry from leader.</param>
/// <param name="ct">Cancellation token.</param>
public Task TryAppendFromLeaderAsync(RaftLogEntry entry, CancellationToken ct) public Task TryAppendFromLeaderAsync(RaftLogEntry entry, CancellationToken ct)
{ {
_ = ct; _ = ct;
@@ -769,6 +910,10 @@ public sealed class RaftNode : IDisposable
return Task.CompletedTask; return Task.CompletedTask;
} }
/// <summary>
/// Creates and persists a snapshot at the current applied index.
/// </summary>
/// <param name="ct">Cancellation token for snapshot persistence.</param>
public async Task<RaftSnapshot> CreateSnapshotAsync(CancellationToken ct) public async Task<RaftSnapshot> CreateSnapshotAsync(CancellationToken ct)
{ {
var snapshot = new RaftSnapshot var snapshot = new RaftSnapshot
@@ -780,6 +925,11 @@ public sealed class RaftNode : IDisposable
return snapshot; return snapshot;
} }
/// <summary>
/// Installs a snapshot as local log baseline and persists snapshot metadata.
/// </summary>
/// <param name="snapshot">Snapshot to install.</param>
/// <param name="ct">Cancellation token for snapshot persistence.</param>
public Task InstallSnapshotAsync(RaftSnapshot snapshot, CancellationToken ct) public Task InstallSnapshotAsync(RaftSnapshot snapshot, CancellationToken ct)
{ {
Log.ReplaceWithSnapshot(snapshot); Log.ReplaceWithSnapshot(snapshot);
@@ -787,6 +937,9 @@ public sealed class RaftNode : IDisposable
return _snapshotStore.SaveAsync(snapshot, ct); return _snapshotStore.SaveAsync(snapshot, ct);
} }
/// <summary>
/// Forces this node to step down to follower state.
/// </summary>
public void RequestStepDown() public void RequestStepDown()
{ {
Role = RaftRole.Follower; Role = RaftRole.Follower;
@@ -796,12 +949,18 @@ public sealed class RaftNode : IDisposable
_leaderSinceUtc = null; _leaderSinceUtc = null;
} }
/// <summary>
/// Returns current log, commit, and applied progress counters.
/// </summary>
public (long Index, long Commit, long Applied) Progress() public (long Index, long Commit, long Applied) Progress()
{ {
var index = Log.Entries.Count > 0 ? Log.Entries[^1].Index : Log.BaseIndex; var index = Log.Entries.Count > 0 ? Log.Entries[^1].Index : Log.BaseIndex;
return (index, CommitIndex, AppliedIndex); return (index, CommitIndex, AppliedIndex);
} }
/// <summary>
/// Returns approximate in-memory log size in entries and UTF-8 command bytes.
/// </summary>
public (long Entries, long Bytes) Size() public (long Entries, long Bytes) Size()
{ {
var entries = (long)Log.Entries.Count; var entries = (long)Log.Entries.Count;
@@ -809,9 +968,16 @@ public sealed class RaftNode : IDisposable
return (entries, bytes); return (entries, bytes);
} }
/// <summary>
/// Returns configured cluster size fallbacking to member count.
/// </summary>
public int ClusterSize() public int ClusterSize()
=> _clusterSize > 0 ? _clusterSize : Math.Max(_members.Count, 1); => _clusterSize > 0 ? _clusterSize : Math.Max(_members.Count, 1);
/// <summary>
/// Adjusts bootstrap cluster size before any leader has been observed.
/// </summary>
/// <param name="clusterSize">Desired bootstrap cluster size.</param>
public bool AdjustBootClusterSize(int clusterSize) public bool AdjustBootClusterSize(int clusterSize)
{ {
if (!Leaderless || HadPreviousLeader) if (!Leaderless || HadPreviousLeader)
@@ -821,6 +987,10 @@ public sealed class RaftNode : IDisposable
return true; return true;
} }
/// <summary>
/// Adjusts active cluster size when this node is current leader.
/// </summary>
/// <param name="clusterSize">Desired cluster size.</param>
public bool AdjustClusterSize(int clusterSize) public bool AdjustClusterSize(int clusterSize)
{ {
if (!IsLeader) if (!IsLeader)
@@ -830,6 +1000,10 @@ public sealed class RaftNode : IDisposable
return true; return true;
} }
/// <summary>
/// Enables or disables observer (non-voting) mode.
/// </summary>
/// <param name="enabled">True to enable observer mode.</param>
public void SetObserver(bool enabled) public void SetObserver(bool enabled)
=> _observerMode = enabled; => _observerMode = enabled;
@@ -853,6 +1027,9 @@ public sealed class RaftNode : IDisposable
return TimeSpan.FromMilliseconds(ms); return TimeSpan.FromMilliseconds(ms);
} }
/// <summary>
/// Returns randomized timeout used for campaign pacing and tests.
/// </summary>
public TimeSpan RandomizedCampaignTimeout() public TimeSpan RandomizedCampaignTimeout()
{ {
var min = (int)MinCampaignTimeoutDefault.TotalMilliseconds; var min = (int)MinCampaignTimeoutDefault.TotalMilliseconds;
@@ -876,6 +1053,7 @@ public sealed class RaftNode : IDisposable
/// an election campaign is triggered automatically. /// an election campaign is triggered automatically.
/// Go reference: raft.go:1500-1550 (campaign logic). /// Go reference: raft.go:1500-1550 (campaign logic).
/// </summary> /// </summary>
/// <param name="ct">Cancellation token that stops timer callbacks.</param>
public void StartElectionTimer(CancellationToken ct = default) public void StartElectionTimer(CancellationToken ct = default)
{ {
_electionTimerCts = CancellationTokenSource.CreateLinkedTokenSource(ct); _electionTimerCts = CancellationTokenSource.CreateLinkedTokenSource(ct);
@@ -918,6 +1096,8 @@ public sealed class RaftNode : IDisposable
/// ///
/// Go reference: raft.go stepDown (leadership transfer variant) / sendTimeoutNow. /// Go reference: raft.go stepDown (leadership transfer variant) / sendTimeoutNow.
/// </summary> /// </summary>
/// <param name="targetId">Peer id to transfer leadership to.</param>
/// <param name="ct">Cancellation token for transfer workflow.</param>
public async Task<bool> TransferLeadershipAsync(string targetId, CancellationToken ct) public async Task<bool> TransferLeadershipAsync(string targetId, CancellationToken ct)
{ {
if (Role != RaftRole.Leader) if (Role != RaftRole.Leader)
@@ -965,6 +1145,7 @@ public sealed class RaftNode : IDisposable
/// ///
/// Go reference: raft.go processTimeoutNow — triggers immediate campaign. /// Go reference: raft.go processTimeoutNow — triggers immediate campaign.
/// </summary> /// </summary>
/// <param name="term">Leader term attached to TimeoutNow.</param>
public void ReceiveTimeoutNow(ulong term) public void ReceiveTimeoutNow(ulong term)
{ {
// Accept the sender's term if it's higher. // Accept the sender's term if it's higher.
@@ -1007,6 +1188,7 @@ public sealed class RaftNode : IDisposable
/// Checks if this node's log is current (within one election timeout of the leader). /// Checks if this node's log is current (within one election timeout of the leader).
/// Go reference: raft.go isCurrent check. /// Go reference: raft.go isCurrent check.
/// </summary> /// </summary>
/// <param name="electionTimeout">Timeout window used for currentness check.</param>
public bool IsCurrent(TimeSpan electionTimeout) public bool IsCurrent(TimeSpan electionTimeout)
{ {
// A leader is always current // A leader is always current
@@ -1020,6 +1202,7 @@ public sealed class RaftNode : IDisposable
/// <summary> /// <summary>
/// Overall health check: node is active and peers are responsive. /// Overall health check: node is active and peers are responsive.
/// </summary> /// </summary>
/// <param name="healthThreshold">Maximum age of peer contact considered healthy.</param>
public bool IsHealthy(TimeSpan healthThreshold) public bool IsHealthy(TimeSpan healthThreshold)
{ {
if (Role == RaftRole.Leader) if (Role == RaftRole.Leader)
@@ -1044,6 +1227,10 @@ public sealed class RaftNode : IDisposable
/// Pre-votes do NOT change any persistent state (no term increment, no votedFor change). /// Pre-votes do NOT change any persistent state (no term increment, no votedFor change).
/// Go reference: raft.go:1600-1700 (pre-vote logic). /// Go reference: raft.go:1600-1700 (pre-vote logic).
/// </summary> /// </summary>
/// <param name="term">Candidate term.</param>
/// <param name="lastTerm">Candidate log last term.</param>
/// <param name="lastIndex">Candidate log last index.</param>
/// <param name="candidateId">Candidate id requesting pre-vote.</param>
public bool RequestPreVote(ulong term, ulong lastTerm, ulong lastIndex, string candidateId) public bool RequestPreVote(ulong term, ulong lastTerm, ulong lastIndex, string candidateId)
{ {
_ = candidateId; // used for logging in production; not needed for correctness _ = candidateId; // used for logging in production; not needed for correctness
@@ -1126,6 +1313,9 @@ public sealed class RaftNode : IDisposable
} }
} }
/// <summary>
/// Stops elections and signals waiters that this node is stopped.
/// </summary>
public void Stop() public void Stop()
{ {
Role = RaftRole.Follower; Role = RaftRole.Follower;
@@ -1135,11 +1325,17 @@ public sealed class RaftNode : IDisposable
_stopSignal.TrySetResult(); _stopSignal.TrySetResult();
} }
/// <summary>
/// Blocks until stop signal has been observed.
/// </summary>
public void WaitForStop() public void WaitForStop()
{ {
_stopSignal.Task.GetAwaiter().GetResult(); _stopSignal.Task.GetAwaiter().GetResult();
} }
/// <summary>
/// Stops the node and deletes persisted RAFT state from disk.
/// </summary>
public void Delete() public void Delete()
{ {
Stop(); Stop();
@@ -1152,6 +1348,10 @@ public sealed class RaftNode : IDisposable
Directory.Delete(_persistDirectory, recursive: true); Directory.Delete(_persistDirectory, recursive: true);
} }
/// <summary>
/// Persists log, applied index, and durable term/vote metadata to storage.
/// </summary>
/// <param name="ct">Cancellation token for I/O operations.</param>
public async Task PersistAsync(CancellationToken ct) public async Task PersistAsync(CancellationToken ct)
{ {
var dir = _persistDirectory ?? Path.Combine(Path.GetTempPath(), "natsdotnet-raft", Id); var dir = _persistDirectory ?? Path.Combine(Path.GetTempPath(), "natsdotnet-raft", Id);
@@ -1172,6 +1372,10 @@ public sealed class RaftNode : IDisposable
ct); ct);
} }
/// <summary>
/// Loads previously persisted log and durable term/vote state from storage.
/// </summary>
/// <param name="ct">Cancellation token for I/O operations.</param>
public async Task LoadPersistedStateAsync(CancellationToken ct) public async Task LoadPersistedStateAsync(CancellationToken ct)
{ {
var dir = _persistDirectory ?? Path.Combine(Path.GetTempPath(), "natsdotnet-raft", Id); var dir = _persistDirectory ?? Path.Combine(Path.GetTempPath(), "natsdotnet-raft", Id);
@@ -1207,10 +1411,19 @@ public sealed class RaftNode : IDisposable
/// <summary>Durable term + vote metadata written alongside the log.</summary> /// <summary>Durable term + vote metadata written alongside the log.</summary>
private sealed class RaftMetaState private sealed class RaftMetaState
{ {
/// <summary>
/// Persisted current term.
/// </summary>
public int CurrentTerm { get; set; } public int CurrentTerm { get; set; }
/// <summary>
/// Persisted voted-for candidate id for current term.
/// </summary>
public string? VotedFor { get; set; } public string? VotedFor { get; set; }
} }
/// <summary>
/// Disposes the node by stopping timers and signaling shutdown.
/// </summary>
public void Dispose() public void Dispose()
{ {
Stop(); Stop();
+75
View File
@@ -2,8 +2,31 @@ namespace NATS.Server.Raft;
public interface IRaftTransport public interface IRaftTransport
{ {
/// <summary>
/// Replicates a log entry from leader to a set of followers.
/// </summary>
/// <param name="leaderId">Leader node id sending the append.</param>
/// <param name="followerIds">Follower node ids targeted for replication.</param>
/// <param name="entry">Log entry to replicate.</param>
/// <param name="ct">Cancellation token.</param>
/// <returns>Per-follower append results.</returns>
Task<IReadOnlyList<AppendResult>> AppendEntriesAsync(string leaderId, IReadOnlyList<string> followerIds, RaftLogEntry entry, CancellationToken ct); Task<IReadOnlyList<AppendResult>> AppendEntriesAsync(string leaderId, IReadOnlyList<string> followerIds, RaftLogEntry entry, CancellationToken ct);
/// <summary>
/// Sends a vote request from a candidate to a voter.
/// </summary>
/// <param name="candidateId">Candidate node id requesting the vote.</param>
/// <param name="voterId">Target voter node id.</param>
/// <param name="request">Vote request payload.</param>
/// <param name="ct">Cancellation token.</param>
/// <returns>Vote response from the target voter.</returns>
Task<VoteResponse> RequestVoteAsync(string candidateId, string voterId, VoteRequest request, CancellationToken ct); Task<VoteResponse> RequestVoteAsync(string candidateId, string voterId, VoteRequest request, CancellationToken ct);
/// <summary>
/// Installs a snapshot from leader to follower.
/// </summary>
/// <param name="leaderId">Leader node id sending the snapshot.</param>
/// <param name="followerId">Follower node id receiving the snapshot.</param>
/// <param name="snapshot">Snapshot payload.</param>
/// <param name="ct">Cancellation token.</param>
Task InstallSnapshotAsync(string leaderId, string followerId, RaftSnapshot snapshot, CancellationToken ct); Task InstallSnapshotAsync(string leaderId, string followerId, RaftSnapshot snapshot, CancellationToken ct);
/// <summary> /// <summary>
@@ -11,6 +34,10 @@ public interface IRaftTransport
/// an election and bypass its election timer. Used for leadership transfer. /// an election and bypass its election timer. Used for leadership transfer.
/// Go reference: raft.go sendTimeoutNow /// Go reference: raft.go sendTimeoutNow
/// </summary> /// </summary>
/// <param name="leaderId">Leader node id issuing TimeoutNow.</param>
/// <param name="targetId">Target follower node id.</param>
/// <param name="term">Leader term to include in the request.</param>
/// <param name="ct">Cancellation token.</param>
Task SendTimeoutNowAsync(string leaderId, string targetId, ulong term, CancellationToken ct); Task SendTimeoutNowAsync(string leaderId, string targetId, ulong term, CancellationToken ct);
/// <summary> /// <summary>
@@ -20,6 +47,11 @@ public interface IRaftTransport
/// serving a linearizable read. /// serving a linearizable read.
/// Go reference: raft.go — leader sends AppendEntries (empty) to confirm quorum for reads. /// Go reference: raft.go — leader sends AppendEntries (empty) to confirm quorum for reads.
/// </summary> /// </summary>
/// <param name="leaderId">Leader node id sending heartbeats.</param>
/// <param name="followerIds">Follower node ids to contact.</param>
/// <param name="term">Leader term for heartbeat messages.</param>
/// <param name="onAck">Callback invoked for each acknowledged follower id.</param>
/// <param name="ct">Cancellation token.</param>
Task SendHeartbeatAsync(string leaderId, IReadOnlyList<string> followerIds, int term, Action<string> onAck, CancellationToken ct); Task SendHeartbeatAsync(string leaderId, IReadOnlyList<string> followerIds, int term, Action<string> onAck, CancellationToken ct);
} }
@@ -27,11 +59,23 @@ public sealed class InMemoryRaftTransport : IRaftTransport
{ {
private readonly Dictionary<string, RaftNode> _nodes = new(StringComparer.Ordinal); private readonly Dictionary<string, RaftNode> _nodes = new(StringComparer.Ordinal);
/// <summary>
/// Registers a node in the in-memory transport registry.
/// </summary>
/// <param name="node">Node instance to register.</param>
public void Register(RaftNode node) public void Register(RaftNode node)
{ {
_nodes[node.Id] = node; _nodes[node.Id] = node;
} }
/// <summary>
/// Replicates a log entry to registered follower nodes.
/// </summary>
/// <param name="leaderId">Leader node id sending the append.</param>
/// <param name="followerIds">Follower node ids targeted for replication.</param>
/// <param name="entry">Log entry to replicate.</param>
/// <param name="ct">Cancellation token.</param>
/// <returns>Per-follower append results.</returns>
public Task<IReadOnlyList<AppendResult>> AppendEntriesAsync(string leaderId, IReadOnlyList<string> followerIds, RaftLogEntry entry, CancellationToken ct) public Task<IReadOnlyList<AppendResult>> AppendEntriesAsync(string leaderId, IReadOnlyList<string> followerIds, RaftLogEntry entry, CancellationToken ct)
{ {
var results = new List<AppendResult>(followerIds.Count); var results = new List<AppendResult>(followerIds.Count);
@@ -51,6 +95,14 @@ public sealed class InMemoryRaftTransport : IRaftTransport
return Task.FromResult<IReadOnlyList<AppendResult>>(results); return Task.FromResult<IReadOnlyList<AppendResult>>(results);
} }
/// <summary>
/// Sends a vote request to a registered voter node.
/// </summary>
/// <param name="candidateId">Candidate node id requesting a vote.</param>
/// <param name="voterId">Voter node id receiving the request.</param>
/// <param name="request">Vote request payload.</param>
/// <param name="ct">Cancellation token.</param>
/// <returns>Vote response from the voter or a rejected response if unreachable.</returns>
public Task<VoteResponse> RequestVoteAsync(string candidateId, string voterId, VoteRequest request, CancellationToken ct) public Task<VoteResponse> RequestVoteAsync(string candidateId, string voterId, VoteRequest request, CancellationToken ct)
{ {
if (_nodes.TryGetValue(voterId, out var node)) if (_nodes.TryGetValue(voterId, out var node))
@@ -59,6 +111,13 @@ public sealed class InMemoryRaftTransport : IRaftTransport
return Task.FromResult(new VoteResponse { Granted = false }); return Task.FromResult(new VoteResponse { Granted = false });
} }
/// <summary>
/// Installs a snapshot on a registered follower node.
/// </summary>
/// <param name="leaderId">Leader node id sending the snapshot.</param>
/// <param name="followerId">Follower node id receiving the snapshot.</param>
/// <param name="snapshot">Snapshot payload.</param>
/// <param name="ct">Cancellation token.</param>
public async Task InstallSnapshotAsync(string leaderId, string followerId, RaftSnapshot snapshot, CancellationToken ct) public async Task InstallSnapshotAsync(string leaderId, string followerId, RaftSnapshot snapshot, CancellationToken ct)
{ {
_ = leaderId; _ = leaderId;
@@ -66,6 +125,13 @@ public sealed class InMemoryRaftTransport : IRaftTransport
await node.InstallSnapshotAsync(snapshot, ct); await node.InstallSnapshotAsync(snapshot, ct);
} }
/// <summary>
/// Delivers heartbeat notifications to follower nodes.
/// </summary>
/// <param name="leaderId">Leader node id sending heartbeats.</param>
/// <param name="followerIds">Follower node ids to notify.</param>
/// <param name="term">Leader term for heartbeat messages.</param>
/// <param name="ct">Cancellation token.</param>
public async Task AppendHeartbeatAsync(string leaderId, IReadOnlyList<string> followerIds, int term, CancellationToken ct) public async Task AppendHeartbeatAsync(string leaderId, IReadOnlyList<string> followerIds, int term, CancellationToken ct)
{ {
_ = leaderId; _ = leaderId;
@@ -85,6 +151,11 @@ public sealed class InMemoryRaftTransport : IRaftTransport
/// Unreachable followers (not registered in the transport) produce no acknowledgement. /// Unreachable followers (not registered in the transport) produce no acknowledgement.
/// Go reference: raft.go — leader sends empty AppendEntries to confirm quorum for reads. /// Go reference: raft.go — leader sends empty AppendEntries to confirm quorum for reads.
/// </summary> /// </summary>
/// <param name="leaderId">Leader node id sending heartbeats.</param>
/// <param name="followerIds">Follower node ids to contact.</param>
/// <param name="term">Leader term for heartbeat messages.</param>
/// <param name="onAck">Callback invoked for each acknowledged follower id.</param>
/// <param name="ct">Cancellation token.</param>
public Task SendHeartbeatAsync(string leaderId, IReadOnlyList<string> followerIds, int term, Action<string> onAck, CancellationToken ct) public Task SendHeartbeatAsync(string leaderId, IReadOnlyList<string> followerIds, int term, Action<string> onAck, CancellationToken ct)
{ {
_ = leaderId; _ = leaderId;
@@ -106,6 +177,10 @@ public sealed class InMemoryRaftTransport : IRaftTransport
/// If the target is not registered (simulating an unreachable peer), does nothing. /// If the target is not registered (simulating an unreachable peer), does nothing.
/// Go reference: raft.go sendTimeoutNow / processTimeoutNow /// Go reference: raft.go sendTimeoutNow / processTimeoutNow
/// </summary> /// </summary>
/// <param name="leaderId">Leader node id issuing TimeoutNow.</param>
/// <param name="targetId">Target follower node id.</param>
/// <param name="term">Leader term for the TimeoutNow request.</param>
/// <param name="ct">Cancellation token.</param>
public Task SendTimeoutNowAsync(string leaderId, string targetId, ulong term, CancellationToken ct) public Task SendTimeoutNowAsync(string leaderId, string targetId, ulong term, CancellationToken ct)
{ {
_ = leaderId; _ = leaderId;
+15
View File
@@ -106,6 +106,7 @@ public readonly record struct RaftVoteRequestWire(
/// if the span is not exactly 32 bytes. /// if the span is not exactly 32 bytes.
/// Go: server/raft.go:4571-4583 — decodeVoteRequest() /// Go: server/raft.go:4571-4583 — decodeVoteRequest()
/// </summary> /// </summary>
/// <param name="msg">Raw VoteRequest payload received from the RAFT transport.</param>
public static RaftVoteRequestWire Decode(ReadOnlySpan<byte> msg) public static RaftVoteRequestWire Decode(ReadOnlySpan<byte> msg)
{ {
if (msg.Length != RaftWireConstants.VoteRequestLen) if (msg.Length != RaftWireConstants.VoteRequestLen)
@@ -156,6 +157,7 @@ public readonly record struct RaftVoteResponseWire(
/// if the span is not exactly 17 bytes. /// if the span is not exactly 17 bytes.
/// Go: server/raft.go:4753-4762 — decodeVoteResponse() /// Go: server/raft.go:4753-4762 — decodeVoteResponse()
/// </summary> /// </summary>
/// <param name="msg">Raw VoteResponse payload received from a peer.</param>
public static RaftVoteResponseWire Decode(ReadOnlySpan<byte> msg) public static RaftVoteResponseWire Decode(ReadOnlySpan<byte> msg)
{ {
if (msg.Length != RaftWireConstants.VoteResponseLen) if (msg.Length != RaftWireConstants.VoteResponseLen)
@@ -252,6 +254,7 @@ public readonly record struct RaftAppendEntryWire(
/// if the buffer is shorter than the minimum header length or malformed. /// if the buffer is shorter than the minimum header length or malformed.
/// Go: server/raft.go:2714-2746 — decodeAppendEntry() /// Go: server/raft.go:2714-2746 — decodeAppendEntry()
/// </summary> /// </summary>
/// <param name="msg">Raw AppendEntry payload containing header and entry data.</param>
public static RaftAppendEntryWire Decode(ReadOnlySpan<byte> msg) public static RaftAppendEntryWire Decode(ReadOnlySpan<byte> msg)
{ {
if (msg.Length < RaftWireConstants.AppendEntryBaseLen) if (msg.Length < RaftWireConstants.AppendEntryBaseLen)
@@ -340,6 +343,7 @@ public readonly record struct RaftAppendEntryResponseWire(
/// if the span is not exactly 25 bytes. /// if the span is not exactly 25 bytes.
/// Go: server/raft.go:2799-2817 — decodeAppendEntryResponse() /// Go: server/raft.go:2799-2817 — decodeAppendEntryResponse()
/// </summary> /// </summary>
/// <param name="msg">Raw AppendEntryResponse payload from a follower.</param>
public static RaftAppendEntryResponseWire Decode(ReadOnlySpan<byte> msg) public static RaftAppendEntryResponseWire Decode(ReadOnlySpan<byte> msg)
{ {
if (msg.Length != RaftWireConstants.AppendEntryResponseLen) if (msg.Length != RaftWireConstants.AppendEntryResponseLen)
@@ -387,6 +391,7 @@ public readonly record struct RaftPreVoteRequestWire(
/// Decodes a PreVoteRequest from a span. Throws <see cref="ArgumentException"/> /// Decodes a PreVoteRequest from a span. Throws <see cref="ArgumentException"/>
/// if the span is not exactly 32 bytes. /// if the span is not exactly 32 bytes.
/// </summary> /// </summary>
/// <param name="msg">Raw PreVoteRequest payload received from a candidate.</param>
public static RaftPreVoteRequestWire Decode(ReadOnlySpan<byte> msg) public static RaftPreVoteRequestWire Decode(ReadOnlySpan<byte> msg)
{ {
if (msg.Length != RaftWireConstants.VoteRequestLen) if (msg.Length != RaftWireConstants.VoteRequestLen)
@@ -429,6 +434,7 @@ public readonly record struct RaftPreVoteResponseWire(
/// Decodes a PreVoteResponse from a span. Throws <see cref="ArgumentException"/> /// Decodes a PreVoteResponse from a span. Throws <see cref="ArgumentException"/>
/// if the span is not exactly 17 bytes. /// if the span is not exactly 17 bytes.
/// </summary> /// </summary>
/// <param name="msg">Raw PreVoteResponse payload received from a peer.</param>
public static RaftPreVoteResponseWire Decode(ReadOnlySpan<byte> msg) public static RaftPreVoteResponseWire Decode(ReadOnlySpan<byte> msg)
{ {
if (msg.Length != RaftWireConstants.VoteResponseLen) if (msg.Length != RaftWireConstants.VoteResponseLen)
@@ -475,6 +481,7 @@ public readonly record struct RaftTimeoutNowWire(ulong Term, string LeaderId)
/// Decodes a TimeoutNow message from a span. Throws <see cref="ArgumentException"/> /// Decodes a TimeoutNow message from a span. Throws <see cref="ArgumentException"/>
/// if the span is not exactly 16 bytes. /// if the span is not exactly 16 bytes.
/// </summary> /// </summary>
/// <param name="msg">Raw TimeoutNow payload sent by the current leader.</param>
public static RaftTimeoutNowWire Decode(ReadOnlySpan<byte> msg) public static RaftTimeoutNowWire Decode(ReadOnlySpan<byte> msg)
{ {
if (msg.Length != MessageLen) if (msg.Length != MessageLen)
@@ -538,6 +545,7 @@ public readonly record struct RaftInstallSnapshotChunkWire(
/// Throws <see cref="ArgumentException"/> when the buffer is shorter than /// Throws <see cref="ArgumentException"/> when the buffer is shorter than
/// the fixed <see cref="HeaderLen"/> bytes. /// the fixed <see cref="HeaderLen"/> bytes.
/// </summary> /// </summary>
/// <param name="msg">Raw InstallSnapshot chunk payload including fixed header and chunk data.</param>
public static RaftInstallSnapshotChunkWire Decode(ReadOnlySpan<byte> msg) public static RaftInstallSnapshotChunkWire Decode(ReadOnlySpan<byte> msg)
{ {
if (msg.Length < HeaderLen) if (msg.Length < HeaderLen)
@@ -566,6 +574,8 @@ internal static class RaftWireHelpers
/// copy(buf[:idLen], id) semantics). /// copy(buf[:idLen], id) semantics).
/// Go: server/raft.go:2693 — copy(buf[:idLen], ae.leader) /// Go: server/raft.go:2693 — copy(buf[:idLen], ae.leader)
/// </summary> /// </summary>
/// <param name="dest">Destination span containing an ID field slot in a RAFT wire frame.</param>
/// <param name="id">Peer/leader identifier to encode.</param>
public static void WriteId(Span<byte> dest, string id) public static void WriteId(Span<byte> dest, string id)
{ {
// Zero-fill the 8-byte slot first. // Zero-fill the 8-byte slot first.
@@ -580,6 +590,7 @@ internal static class RaftWireHelpers
/// that zero-padded IDs decode back to their original string. /// that zero-padded IDs decode back to their original string.
/// Go: server/raft.go:4581 — string(copyBytes(msg[24:24+idLen])) /// Go: server/raft.go:4581 — string(copyBytes(msg[24:24+idLen]))
/// </summary> /// </summary>
/// <param name="src">Source span containing an encoded fixed-width ID field.</param>
public static string ReadId(ReadOnlySpan<byte> src) public static string ReadId(ReadOnlySpan<byte> src)
{ {
var idBytes = src[..RaftWireConstants.IdLen]; var idBytes = src[..RaftWireConstants.IdLen];
@@ -594,6 +605,8 @@ internal static class RaftWireHelpers
/// number of bytes written (1-10). /// number of bytes written (1-10).
/// Go: server/raft.go:2682 — binary.PutUvarint(_lterm[:], ae.lterm) /// Go: server/raft.go:2682 — binary.PutUvarint(_lterm[:], ae.lterm)
/// </summary> /// </summary>
/// <param name="buf">Destination buffer where the uvarint bytes are written.</param>
/// <param name="value">Unsigned integer value to encode.</param>
public static int WriteUvarint(Span<byte> buf, ulong value) public static int WriteUvarint(Span<byte> buf, ulong value)
{ {
var pos = 0; var pos = 0;
@@ -611,6 +624,8 @@ internal static class RaftWireHelpers
/// and returns the number of bytes consumed (0 on overflow or empty input). /// and returns the number of bytes consumed (0 on overflow or empty input).
/// Go: server/raft.go:2740 — binary.Uvarint(msg[ri:]) /// Go: server/raft.go:2740 — binary.Uvarint(msg[ri:])
/// </summary> /// </summary>
/// <param name="buf">Source buffer containing uvarint-encoded data.</param>
/// <param name="value">Decoded unsigned integer output.</param>
public static int ReadUvarint(ReadOnlySpan<byte> buf, out ulong value) public static int ReadUvarint(ReadOnlySpan<byte> buf, out ulong value)
{ {
value = 0; value = 0;
+107
View File
@@ -17,7 +17,9 @@ public sealed class RouteConnection(Socket socket) : IAsyncDisposable
private readonly CancellationTokenSource _closedCts = new(); private readonly CancellationTokenSource _closedCts = new();
private Task? _frameLoopTask; private Task? _frameLoopTask;
/// <summary>Remote server id learned during ROUTE handshake.</summary>
public string? RemoteServerId { get; private set; } public string? RemoteServerId { get; private set; }
/// <summary>Remote endpoint string for diagnostics and monitoring.</summary>
public string RemoteEndpoint => _socket.RemoteEndPoint?.ToString() ?? Guid.NewGuid().ToString("N"); public string RemoteEndpoint => _socket.RemoteEndPoint?.ToString() ?? Guid.NewGuid().ToString("N");
/// <summary> /// <summary>
@@ -61,6 +63,9 @@ public sealed class RouteConnection(Socket socket) : IAsyncDisposable
/// either side is 0 for backward compatibility with peers that do not support pooling. /// either side is 0 for backward compatibility with peers that do not support pooling.
/// Go reference: server/route.go negotiateRoutePool. /// Go reference: server/route.go negotiateRoutePool.
/// </summary> /// </summary>
/// <param name="localPoolSize">Local configured route pool size.</param>
/// <param name="remotePoolSize">Remote configured route pool size.</param>
/// <returns>Negotiated pool size.</returns>
public static int NegotiatePoolSize(int localPoolSize, int remotePoolSize) public static int NegotiatePoolSize(int localPoolSize, int remotePoolSize)
{ {
if (localPoolSize == 0 || remotePoolSize == 0) if (localPoolSize == 0 || remotePoolSize == 0)
@@ -72,14 +77,22 @@ public sealed class RouteConnection(Socket socket) : IAsyncDisposable
/// <summary> /// <summary>
/// Applies the result of pool size negotiation to this connection. /// Applies the result of pool size negotiation to this connection.
/// </summary> /// </summary>
/// <param name="negotiatedPoolSize">Negotiated pool size.</param>
internal void SetNegotiatedPoolSize(int negotiatedPoolSize) internal void SetNegotiatedPoolSize(int negotiatedPoolSize)
{ {
NegotiatedPoolSize = negotiatedPoolSize; NegotiatedPoolSize = negotiatedPoolSize;
} }
/// <summary>Callback invoked when remote RS/LS interest updates are received.</summary>
public Func<RemoteSubscription, Task>? RemoteSubscriptionReceived { get; set; } public Func<RemoteSubscription, Task>? RemoteSubscriptionReceived { get; set; }
/// <summary>Callback invoked when remote RMSG payloads are received.</summary>
public Func<RouteMessage, Task>? RoutedMessageReceived { get; set; } public Func<RouteMessage, Task>? RoutedMessageReceived { get; set; }
/// <summary>
/// Performs outbound ROUTE handshake by sending local id and reading remote id.
/// </summary>
/// <param name="serverId">Local server id.</param>
/// <param name="ct">Cancellation token for I/O operations.</param>
public async Task PerformOutboundHandshakeAsync(string serverId, CancellationToken ct) public async Task PerformOutboundHandshakeAsync(string serverId, CancellationToken ct)
{ {
await WriteLineAsync($"ROUTE {serverId}", ct); await WriteLineAsync($"ROUTE {serverId}", ct);
@@ -87,6 +100,11 @@ public sealed class RouteConnection(Socket socket) : IAsyncDisposable
RemoteServerId = ParseHandshake(line); RemoteServerId = ParseHandshake(line);
} }
/// <summary>
/// Performs inbound ROUTE handshake by reading remote id and sending local id.
/// </summary>
/// <param name="serverId">Local server id.</param>
/// <param name="ct">Cancellation token for I/O operations.</param>
public async Task PerformInboundHandshakeAsync(string serverId, CancellationToken ct) public async Task PerformInboundHandshakeAsync(string serverId, CancellationToken ct)
{ {
var line = await ReadLineAsync(ct); var line = await ReadLineAsync(ct);
@@ -94,6 +112,10 @@ public sealed class RouteConnection(Socket socket) : IAsyncDisposable
await WriteLineAsync($"ROUTE {serverId}", ct); await WriteLineAsync($"ROUTE {serverId}", ct);
} }
/// <summary>
/// Starts the background route frame read loop.
/// </summary>
/// <param name="ct">Cancellation token controlling loop lifetime.</param>
public void StartFrameLoop(CancellationToken ct) public void StartFrameLoop(CancellationToken ct)
{ {
if (_frameLoopTask != null) if (_frameLoopTask != null)
@@ -103,9 +125,24 @@ public sealed class RouteConnection(Socket socket) : IAsyncDisposable
_frameLoopTask = Task.Run(() => ReadFramesAsync(linked.Token), linked.Token); _frameLoopTask = Task.Run(() => ReadFramesAsync(linked.Token), linked.Token);
} }
/// <summary>
/// Sends an RS+ protocol line for route interest propagation.
/// </summary>
/// <param name="account">Account associated with the interest update.</param>
/// <param name="subject">Subject being added.</param>
/// <param name="queue">Optional queue group.</param>
/// <param name="ct">Cancellation token for I/O operations.</param>
public async Task SendRsPlusAsync(string account, string subject, string? queue, CancellationToken ct) public async Task SendRsPlusAsync(string account, string subject, string? queue, CancellationToken ct)
=> await SendRsPlusAsync(account, subject, queue, queueWeight: 0, ct); => await SendRsPlusAsync(account, subject, queue, queueWeight: 0, ct);
/// <summary>
/// Sends an RS+ protocol line with optional queue weight.
/// </summary>
/// <param name="account">Account associated with the interest update.</param>
/// <param name="subject">Subject being added.</param>
/// <param name="queue">Optional queue group.</param>
/// <param name="queueWeight">Queue weight to advertise when queue is present.</param>
/// <param name="ct">Cancellation token for I/O operations.</param>
public async Task SendRsPlusAsync(string account, string subject, string? queue, int queueWeight, CancellationToken ct) public async Task SendRsPlusAsync(string account, string subject, string? queue, int queueWeight, CancellationToken ct)
{ {
string frame; string frame;
@@ -119,6 +156,13 @@ public sealed class RouteConnection(Socket socket) : IAsyncDisposable
await WriteLineAsync(frame, ct); await WriteLineAsync(frame, ct);
} }
/// <summary>
/// Sends an RS- protocol line to remove route interest.
/// </summary>
/// <param name="account">Account associated with the interest update.</param>
/// <param name="subject">Subject being removed.</param>
/// <param name="queue">Optional queue group.</param>
/// <param name="ct">Cancellation token for I/O operations.</param>
public async Task SendRsMinusAsync(string account, string subject, string? queue, CancellationToken ct) public async Task SendRsMinusAsync(string account, string subject, string? queue, CancellationToken ct)
{ {
var frame = queue is { Length: > 0 } var frame = queue is { Length: > 0 }
@@ -127,6 +171,13 @@ public sealed class RouteConnection(Socket socket) : IAsyncDisposable
await WriteLineAsync(frame, ct); await WriteLineAsync(frame, ct);
} }
/// <summary>
/// Sends an LS+ protocol line for leaf interest propagation over route links.
/// </summary>
/// <param name="account">Account associated with the interest update.</param>
/// <param name="subject">Subject being added.</param>
/// <param name="queue">Optional queue group.</param>
/// <param name="ct">Cancellation token for I/O operations.</param>
public async Task SendLsPlusAsync(string account, string subject, string? queue, CancellationToken ct) public async Task SendLsPlusAsync(string account, string subject, string? queue, CancellationToken ct)
{ {
var frame = queue is { Length: > 0 } var frame = queue is { Length: > 0 }
@@ -135,6 +186,13 @@ public sealed class RouteConnection(Socket socket) : IAsyncDisposable
await WriteLineAsync(frame, ct); await WriteLineAsync(frame, ct);
} }
/// <summary>
/// Sends an LS- protocol line for leaf interest removal over route links.
/// </summary>
/// <param name="account">Account associated with the interest update.</param>
/// <param name="subject">Subject being removed.</param>
/// <param name="queue">Optional queue group.</param>
/// <param name="ct">Cancellation token for I/O operations.</param>
public async Task SendLsMinusAsync(string account, string subject, string? queue, CancellationToken ct) public async Task SendLsMinusAsync(string account, string subject, string? queue, CancellationToken ct)
{ {
var frame = queue is { Length: > 0 } var frame = queue is { Length: > 0 }
@@ -143,6 +201,11 @@ public sealed class RouteConnection(Socket socket) : IAsyncDisposable
await WriteLineAsync(frame, ct); await WriteLineAsync(frame, ct);
} }
/// <summary>
/// Sends a batch of RS+ protocol lines for non-removal subscriptions.
/// </summary>
/// <param name="subscriptions">Subscriptions to encode and send.</param>
/// <param name="ct">Cancellation token for I/O operations.</param>
public async Task SendRouteSubProtosAsync(IEnumerable<RemoteSubscription> subscriptions, CancellationToken ct) public async Task SendRouteSubProtosAsync(IEnumerable<RemoteSubscription> subscriptions, CancellationToken ct)
{ {
var protos = new List<string>(); var protos = new List<string>();
@@ -162,6 +225,11 @@ public sealed class RouteConnection(Socket socket) : IAsyncDisposable
await SendRouteSubOrUnSubProtosAsync(protos, ct); await SendRouteSubOrUnSubProtosAsync(protos, ct);
} }
/// <summary>
/// Sends a batch of RS- protocol lines for subscription removals.
/// </summary>
/// <param name="subscriptions">Subscriptions to encode and send as removals.</param>
/// <param name="ct">Cancellation token for I/O operations.</param>
public async Task SendRouteUnSubProtosAsync(IEnumerable<RemoteSubscription> subscriptions, CancellationToken ct) public async Task SendRouteUnSubProtosAsync(IEnumerable<RemoteSubscription> subscriptions, CancellationToken ct)
{ {
var protos = new List<string>(); var protos = new List<string>();
@@ -176,6 +244,11 @@ public sealed class RouteConnection(Socket socket) : IAsyncDisposable
await SendRouteSubOrUnSubProtosAsync(protos, ct); await SendRouteSubOrUnSubProtosAsync(protos, ct);
} }
/// <summary>
/// Sends a pre-built batch of route subscription protocol lines.
/// </summary>
/// <param name="protocols">Protocol lines to send.</param>
/// <param name="ct">Cancellation token for I/O operations.</param>
public async Task SendRouteSubOrUnSubProtosAsync(IEnumerable<string> protocols, CancellationToken ct) public async Task SendRouteSubOrUnSubProtosAsync(IEnumerable<string> protocols, CancellationToken ct)
{ {
var sb = new StringBuilder(); var sb = new StringBuilder();
@@ -202,6 +275,14 @@ public sealed class RouteConnection(Socket socket) : IAsyncDisposable
} }
} }
/// <summary>
/// Sends an RMSG payload frame to the remote route.
/// </summary>
/// <param name="account">Account associated with the message.</param>
/// <param name="subject">Subject being routed.</param>
/// <param name="replyTo">Optional reply subject.</param>
/// <param name="payload">Payload bytes.</param>
/// <param name="ct">Cancellation token for I/O operations.</param>
public async Task SendRmsgAsync(string account, string subject, string? replyTo, ReadOnlyMemory<byte> payload, CancellationToken ct) public async Task SendRmsgAsync(string account, string subject, string? replyTo, ReadOnlyMemory<byte> payload, CancellationToken ct)
{ {
var replyToken = string.IsNullOrEmpty(replyTo) ? "-" : replyTo; var replyToken = string.IsNullOrEmpty(replyTo) ? "-" : replyTo;
@@ -221,6 +302,10 @@ public sealed class RouteConnection(Socket socket) : IAsyncDisposable
} }
} }
/// <summary>
/// Waits until the route frame loop exits.
/// </summary>
/// <param name="ct">Cancellation token for wait operation.</param>
public async Task WaitUntilClosedAsync(CancellationToken ct) public async Task WaitUntilClosedAsync(CancellationToken ct)
{ {
if (_frameLoopTask == null) if (_frameLoopTask == null)
@@ -230,6 +315,9 @@ public sealed class RouteConnection(Socket socket) : IAsyncDisposable
await _frameLoopTask.WaitAsync(linked.Token); await _frameLoopTask.WaitAsync(linked.Token);
} }
/// <summary>
/// Disposes this route connection and stops background processing.
/// </summary>
public async ValueTask DisposeAsync() public async ValueTask DisposeAsync()
{ {
await _closedCts.CancelAsync(); await _closedCts.CancelAsync();
@@ -413,6 +501,14 @@ public sealed class RouteConnection(Socket socket) : IAsyncDisposable
return true; return true;
} }
/// <summary>
/// Parses a remote RS-/LS- protocol line into account/subject/queue components.
/// </summary>
/// <param name="line">Raw protocol line.</param>
/// <param name="account">Receives parsed account on success.</param>
/// <param name="subject">Receives parsed subject on success.</param>
/// <param name="queue">Receives parsed queue group on success.</param>
/// <returns><see langword="true"/> when parsing succeeds.</returns>
internal static bool TryParseRemoteUnsub(string line, out string account, out string subject, out string? queue) internal static bool TryParseRemoteUnsub(string line, out string account, out string subject, out string? queue)
{ {
account = "$G"; account = "$G";
@@ -426,6 +522,10 @@ public sealed class RouteConnection(Socket socket) : IAsyncDisposable
return TryParseAccountScopedInterest(parts, out account, out subject, out queue); return TryParseAccountScopedInterest(parts, out account, out subject, out queue);
} }
/// <summary>
/// Indicates whether this route was solicited outbound by this server.
/// </summary>
/// <returns><see langword="true"/> when route is solicited.</returns>
public bool IsSolicitedRoute() public bool IsSolicitedRoute()
=> IsSolicited; => IsSolicited;
@@ -434,6 +534,13 @@ public sealed class RouteConnection(Socket socket) : IAsyncDisposable
|| token.Contains('*', StringComparison.Ordinal) || token.Contains('*', StringComparison.Ordinal)
|| token.Contains('>', StringComparison.Ordinal); || token.Contains('>', StringComparison.Ordinal);
/// <summary>
/// Builds the CONNECT payload JSON used during route handshake.
/// </summary>
/// <param name="serverId">Local server id and advertised name.</param>
/// <param name="accounts">Optional exported account list.</param>
/// <param name="topologySnapshot">Optional topology snapshot string.</param>
/// <returns>Serialized CONNECT JSON payload.</returns>
public static string BuildConnectInfoJson(string serverId, IEnumerable<string>? accounts, string? topologySnapshot) public static string BuildConnectInfoJson(string serverId, IEnumerable<string>? accounts, string? topologySnapshot)
{ {
var payload = new var payload = new
@@ -6,6 +6,11 @@ public static class SubjectMatch
public const char Fwc = '>'; // full wildcard public const char Fwc = '>'; // full wildcard
public const char Sep = '.'; // token separator public const char Sep = '.'; // token separator
/// <summary>
/// Validates a subject expression for subscription indexing and routing.
/// </summary>
/// <param name="subject">Subject text to validate.</param>
/// <returns><see langword="true"/> when the subject follows token and wildcard placement rules.</returns>
public static bool IsValidSubject(string subject) public static bool IsValidSubject(string subject)
{ {
if (string.IsNullOrEmpty(subject)) if (string.IsNullOrEmpty(subject))
@@ -47,6 +52,11 @@ public static class SubjectMatch
return true; return true;
} }
/// <summary>
/// Determines whether a subject contains only literal tokens.
/// </summary>
/// <param name="subject">Subject to inspect.</param>
/// <returns><see langword="true"/> when no wildcard token is present.</returns>
public static bool IsLiteral(string subject) public static bool IsLiteral(string subject)
{ {
for (int i = 0; i < subject.Length; i++) for (int i = 0; i < subject.Length; i++)
@@ -64,18 +74,32 @@ public static class SubjectMatch
return true; return true;
} }
/// <summary>
/// Validates a publish subject, which must be syntactically valid and wildcard-free.
/// </summary>
/// <param name="subject">Publish subject to validate.</param>
public static bool IsValidPublishSubject(string subject) public static bool IsValidPublishSubject(string subject)
{ {
return IsValidSubject(subject) && IsLiteral(subject); return IsValidSubject(subject) && IsLiteral(subject);
} }
/// <summary>
/// Checks whether the subject contains wildcard tokens.
/// </summary>
/// <param name="subject">Subject expression to evaluate.</param>
public static bool SubjectHasWildcard(string subject) => !IsLiteral(subject); public static bool SubjectHasWildcard(string subject) => !IsLiteral(subject);
/// <summary>
/// Validates a literal subject used by publish and direct lookup paths.
/// </summary>
/// <param name="subject">Literal subject candidate.</param>
public static bool IsValidLiteralSubject(string subject) => IsValidPublishSubject(subject); public static bool IsValidLiteralSubject(string subject) => IsValidPublishSubject(subject);
/// <summary> /// <summary>
/// Match a literal subject against a pattern that may contain wildcards. /// Match a literal subject against a pattern that may contain wildcards.
/// </summary> /// </summary>
/// <param name="literal">Concrete subject from a published message.</param>
/// <param name="pattern">Subscription pattern that may include <c>*</c> and <c>&gt;</c>.</param>
public static bool MatchLiteral(string literal, string pattern) public static bool MatchLiteral(string literal, string pattern)
{ {
int li = 0, pi = 0; int li = 0, pi = 0;
@@ -119,6 +143,7 @@ public static class SubjectMatch
} }
/// <summary>Count dot-delimited tokens. Empty string returns 0.</summary> /// <summary>Count dot-delimited tokens. Empty string returns 0.</summary>
/// <param name="subject">Subject string to tokenize.</param>
public static int NumTokens(string subject) public static int NumTokens(string subject)
{ {
if (string.IsNullOrEmpty(subject)) if (string.IsNullOrEmpty(subject))
@@ -133,6 +158,8 @@ public static class SubjectMatch
} }
/// <summary>Return the 0-based nth token as a span. Returns empty if out of range.</summary> /// <summary>Return the 0-based nth token as a span. Returns empty if out of range.</summary>
/// <param name="subject">Subject containing dot-delimited tokens.</param>
/// <param name="index">Zero-based token index.</param>
public static ReadOnlySpan<char> TokenAt(string subject, int index) public static ReadOnlySpan<char> TokenAt(string subject, int index)
{ {
if (string.IsNullOrEmpty(subject)) if (string.IsNullOrEmpty(subject))
@@ -160,6 +187,8 @@ public static class SubjectMatch
/// Determines if two subject patterns (possibly containing wildcards) can both /// Determines if two subject patterns (possibly containing wildcards) can both
/// match the same literal subject. Reference: Go sublist.go SubjectsCollide. /// match the same literal subject. Reference: Go sublist.go SubjectsCollide.
/// </summary> /// </summary>
/// <param name="subj1">First subject pattern.</param>
/// <param name="subj2">Second subject pattern.</param>
public static bool SubjectsCollide(string subj1, string subj2) public static bool SubjectsCollide(string subj1, string subj2)
{ {
if (subj1 == subj2) if (subj1 == subj2)
@@ -202,20 +231,40 @@ public static class SubjectMatch
// Go reference: sublist.go SubjectMatchesFilter / subjectIsSubsetMatch / isSubsetMatch / isSubsetMatchTokenized. // Go reference: sublist.go SubjectMatchesFilter / subjectIsSubsetMatch / isSubsetMatch / isSubsetMatchTokenized.
// This is used by JetStream stores to evaluate subject filters with wildcard semantics. // This is used by JetStream stores to evaluate subject filters with wildcard semantics.
/// <summary>
/// Evaluates whether a stream subject satisfies a JetStream filter expression.
/// </summary>
/// <param name="subject">Subject from a stored or incoming message.</param>
/// <param name="filter">Configured filter expression.</param>
public static bool SubjectMatchesFilter(string subject, string filter) => SubjectIsSubsetMatch(subject, filter); public static bool SubjectMatchesFilter(string subject, string filter) => SubjectIsSubsetMatch(subject, filter);
/// <summary>
/// Determines if the left-hand subject pattern is covered by the test pattern.
/// </summary>
/// <param name="subject">Subject pattern being tested.</param>
/// <param name="test">Pattern used as the subset constraint.</param>
public static bool SubjectIsSubsetMatch(string subject, string test) public static bool SubjectIsSubsetMatch(string subject, string test)
{ {
var subjectTokens = TokenizeSubject(subject); var subjectTokens = TokenizeSubject(subject);
return IsSubsetMatch(subjectTokens, test); return IsSubsetMatch(subjectTokens, test);
} }
/// <summary>
/// Determines subset matching for a tokenized subject against a test expression.
/// </summary>
/// <param name="tokens">Tokenized subject to evaluate.</param>
/// <param name="test">Test expression with optional wildcards.</param>
public static bool IsSubsetMatch(string[] tokens, string test) public static bool IsSubsetMatch(string[] tokens, string test)
{ {
var testTokens = TokenizeSubject(test); var testTokens = TokenizeSubject(test);
return IsSubsetMatchTokenized(tokens, testTokens); return IsSubsetMatchTokenized(tokens, testTokens);
} }
/// <summary>
/// Performs token-level subset matching between two subject expressions.
/// </summary>
/// <param name="tokens">Tokenized subject expression.</param>
/// <param name="test">Tokenized test expression.</param>
public static bool IsSubsetMatchTokenized(IReadOnlyList<string> tokens, IReadOnlyList<string> test) public static bool IsSubsetMatchTokenized(IReadOnlyList<string> tokens, IReadOnlyList<string> test)
{ {
for (var i = 0; i < test.Count; i++) for (var i = 0; i < test.Count; i++)
@@ -249,6 +298,11 @@ public static class SubjectMatch
return tokens.Count == test.Count; return tokens.Count == test.Count;
} }
/// <summary>
/// Compares a token span to a candidate token using ordinal semantics.
/// </summary>
/// <param name="token">Token span from a subject.</param>
/// <param name="candidate">Candidate token text.</param>
internal static bool TokenEquals(ReadOnlySpan<char> token, string candidate) internal static bool TokenEquals(ReadOnlySpan<char> token, string candidate)
=> token.SequenceEqual(candidate); => token.SequenceEqual(candidate);
@@ -266,6 +320,8 @@ public static class SubjectMatch
/// <summary> /// <summary>
/// Validates subject. When checkRunes is true, also rejects null bytes. /// Validates subject. When checkRunes is true, also rejects null bytes.
/// </summary> /// </summary>
/// <param name="subject">Subject text to validate.</param>
/// <param name="checkRunes">Whether to reject invalid rune content such as null bytes.</param>
public static bool IsValidSubject(string subject, bool checkRunes) public static bool IsValidSubject(string subject, bool checkRunes)
{ {
if (!IsValidSubject(subject)) if (!IsValidSubject(subject))
+32
View File
@@ -39,6 +39,11 @@ public static class StatusAssertionMaps
[2] = StatusAssertion.Unknown, [2] = StatusAssertion.Unknown,
}; };
/// <summary>
/// Converts numeric status assertion values to their canonical string form.
/// </summary>
/// <param name="sa">Numeric status assertion value.</param>
/// <returns>Status assertion string (defaults to <c>unknown</c>).</returns>
public static string GetStatusAssertionStr(int sa) public static string GetStatusAssertionStr(int sa)
{ {
var value = StatusAssertionIntToVal.TryGetValue(sa, out var mapped) var value = StatusAssertionIntToVal.TryGetValue(sa, out var mapped)
@@ -50,6 +55,7 @@ public static class StatusAssertionMaps
public sealed class StatusAssertionJsonConverter : JsonConverter<StatusAssertion> public sealed class StatusAssertionJsonConverter : JsonConverter<StatusAssertion>
{ {
/// <inheritdoc />
public override StatusAssertion Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) public override StatusAssertion Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
{ {
if (reader.TokenType == JsonTokenType.String) if (reader.TokenType == JsonTokenType.String)
@@ -70,6 +76,7 @@ public sealed class StatusAssertionJsonConverter : JsonConverter<StatusAssertion
return StatusAssertion.Unknown; return StatusAssertion.Unknown;
} }
/// <inheritdoc />
public override void Write(Utf8JsonWriter writer, StatusAssertion value, JsonSerializerOptions options) public override void Write(Utf8JsonWriter writer, StatusAssertion value, JsonSerializerOptions options)
{ {
if (!StatusAssertionMaps.StatusAssertionValToStr.TryGetValue(value, out var str)) if (!StatusAssertionMaps.StatusAssertionValToStr.TryGetValue(value, out var str))
@@ -80,29 +87,38 @@ public sealed class StatusAssertionJsonConverter : JsonConverter<StatusAssertion
public sealed class ChainLink public sealed class ChainLink
{ {
/// <summary>Leaf certificate for chain/OCSP evaluation.</summary>
public X509Certificate2? Leaf { get; set; } public X509Certificate2? Leaf { get; set; }
/// <summary>Issuer certificate corresponding to <see cref="Leaf"/>.</summary>
public X509Certificate2? Issuer { get; set; } public X509Certificate2? Issuer { get; set; }
/// <summary>Discovered HTTP(S) OCSP responder endpoints for the leaf certificate.</summary>
public IReadOnlyList<Uri>? OCSPWebEndpoints { get; set; } public IReadOnlyList<Uri>? OCSPWebEndpoints { get; set; }
} }
public sealed class OcspResponseInfo public sealed class OcspResponseInfo
{ {
/// <summary>Time at which OCSP response status was known to be correct.</summary>
public DateTime ThisUpdate { get; init; } public DateTime ThisUpdate { get; init; }
/// <summary>Optional time after which responder no longer vouches for response status.</summary>
public DateTime? NextUpdate { get; init; } public DateTime? NextUpdate { get; init; }
} }
public sealed class CertInfo public sealed class CertInfo
{ {
[JsonPropertyName("subject")] [JsonPropertyName("subject")]
/// <summary>Subject distinguished name.</summary>
public string Subject { get; init; } = string.Empty; public string Subject { get; init; } = string.Empty;
[JsonPropertyName("issuer")] [JsonPropertyName("issuer")]
/// <summary>Issuer distinguished name.</summary>
public string Issuer { get; init; } = string.Empty; public string Issuer { get; init; } = string.Empty;
[JsonPropertyName("fingerprint")] [JsonPropertyName("fingerprint")]
/// <summary>Certificate fingerprint string.</summary>
public string Fingerprint { get; init; } = string.Empty; public string Fingerprint { get; init; } = string.Empty;
[JsonPropertyName("raw")] [JsonPropertyName("raw")]
/// <summary>Raw DER certificate bytes.</summary>
public byte[] Raw { get; init; } = []; public byte[] Raw { get; init; } = [];
} }
@@ -112,16 +128,32 @@ public sealed class OCSPPeerConfig
public static readonly TimeSpan DefaultOCSPResponderTimeout = TimeSpan.FromSeconds(2); public static readonly TimeSpan DefaultOCSPResponderTimeout = TimeSpan.FromSeconds(2);
public static readonly TimeSpan DefaultTTLUnsetNextUpdate = TimeSpan.FromHours(1); public static readonly TimeSpan DefaultTTLUnsetNextUpdate = TimeSpan.FromHours(1);
/// <summary>Enables OCSP peer verification.</summary>
public bool Verify { get; set; } public bool Verify { get; set; }
/// <summary>Responder timeout in seconds.</summary>
public double Timeout { get; set; } = DefaultOCSPResponderTimeout.TotalSeconds; public double Timeout { get; set; } = DefaultOCSPResponderTimeout.TotalSeconds;
/// <summary>Allowed clock skew in seconds when validating response times.</summary>
public double ClockSkew { get; set; } = DefaultAllowedClockSkew.TotalSeconds; public double ClockSkew { get; set; } = DefaultAllowedClockSkew.TotalSeconds;
/// <summary>When true, OCSP failures are reported as warnings only.</summary>
public bool WarnOnly { get; set; } public bool WarnOnly { get; set; }
/// <summary>When true, unknown OCSP status is treated as success.</summary>
public bool UnknownIsGood { get; set; } public bool UnknownIsGood { get; set; }
/// <summary>When true, allows handshake when CA responder is unreachable.</summary>
public bool AllowWhenCAUnreachable { get; set; } public bool AllowWhenCAUnreachable { get; set; }
/// <summary>Fallback TTL in seconds when OCSP NextUpdate is absent.</summary>
public double TTLUnsetNextUpdate { get; set; } = DefaultTTLUnsetNextUpdate.TotalSeconds; public double TTLUnsetNextUpdate { get; set; } = DefaultTTLUnsetNextUpdate.TotalSeconds;
/// <summary>
/// Creates a default OCSP peer configuration instance.
/// </summary>
/// <returns>Default-configured <see cref="OCSPPeerConfig"/>.</returns>
public static OCSPPeerConfig NewOCSPPeerConfig() => new(); public static OCSPPeerConfig NewOCSPPeerConfig() => new();
/// <summary>
/// Parses OCSP peer configuration values from configuration map entries.
/// </summary>
/// <param name="values">Configuration key/value pairs.</param>
/// <returns>Parsed OCSP peer configuration.</returns>
public static OCSPPeerConfig Parse(IReadOnlyDictionary<string, object?> values) public static OCSPPeerConfig Parse(IReadOnlyDictionary<string, object?> values)
{ {
var cfg = NewOCSPPeerConfig(); var cfg = NewOCSPPeerConfig();
+5
View File
@@ -59,10 +59,15 @@ public sealed class PeekableStream : Stream
public override Task FlushAsync(CancellationToken ct) => _inner.FlushAsync(ct); public override Task FlushAsync(CancellationToken ct) => _inner.FlushAsync(ct);
// Required Stream overrides // Required Stream overrides
/// <inheritdoc />
public override bool CanRead => _inner.CanRead; public override bool CanRead => _inner.CanRead;
/// <inheritdoc />
public override bool CanSeek => false; public override bool CanSeek => false;
/// <inheritdoc />
public override bool CanWrite => _inner.CanWrite; public override bool CanWrite => _inner.CanWrite;
/// <inheritdoc />
public override long Length => throw new NotSupportedException(); public override long Length => throw new NotSupportedException();
/// <inheritdoc />
public override long Position { get => throw new NotSupportedException(); set => throw new NotSupportedException(); } public override long Position { get => throw new NotSupportedException(); set => throw new NotSupportedException(); }
public override long Seek(long offset, SeekOrigin origin) => throw new NotSupportedException(); public override long Seek(long offset, SeekOrigin origin) => throw new NotSupportedException();
public override void SetLength(long value) => throw new NotSupportedException(); public override void SetLength(long value) => throw new NotSupportedException();
+66
View File
@@ -13,6 +13,12 @@ public static class TlsHelper
private const string OcspAccessMethodOid = "1.3.6.1.5.5.7.48.1"; private const string OcspAccessMethodOid = "1.3.6.1.5.5.7.48.1";
private const string OcspSigningEkuOid = "1.3.6.1.5.5.7.3.9"; private const string OcspSigningEkuOid = "1.3.6.1.5.5.7.3.9";
/// <summary>
/// Loads the server certificate, optionally combining with a PEM private key.
/// </summary>
/// <param name="certPath">Path to certificate file.</param>
/// <param name="keyPath">Optional path to private key PEM.</param>
/// <returns>Loaded X509 certificate.</returns>
public static X509Certificate2 LoadCertificate(string certPath, string? keyPath) public static X509Certificate2 LoadCertificate(string certPath, string? keyPath)
{ {
if (keyPath != null) if (keyPath != null)
@@ -20,6 +26,11 @@ public static class TlsHelper
return X509CertificateLoader.LoadCertificateFromFile(certPath); return X509CertificateLoader.LoadCertificateFromFile(certPath);
} }
/// <summary>
/// Loads one or more CA certificates from a PEM file.
/// </summary>
/// <param name="caPath">Path to PEM CA certificate bundle.</param>
/// <returns>Loaded CA certificates.</returns>
public static X509Certificate2Collection LoadCaCertificates(string caPath) public static X509Certificate2Collection LoadCaCertificates(string caPath)
{ {
var pem = File.ReadAllText(caPath); var pem = File.ReadAllText(caPath);
@@ -30,6 +41,8 @@ public static class TlsHelper
/// Parses one or more PEM blocks and requires all blocks to be CERTIFICATE. /// Parses one or more PEM blocks and requires all blocks to be CERTIFICATE.
/// Mirrors Go parseCertPEM behavior by rejecting unexpected block types. /// Mirrors Go parseCertPEM behavior by rejecting unexpected block types.
/// </summary> /// </summary>
/// <param name="pemData">PEM text containing certificate blocks.</param>
/// <returns>Parsed certificate collection.</returns>
public static X509Certificate2Collection ParseCertPem(string pemData) public static X509Certificate2Collection ParseCertPem(string pemData)
{ {
if (string.IsNullOrWhiteSpace(pemData)) if (string.IsNullOrWhiteSpace(pemData))
@@ -66,6 +79,11 @@ public static class TlsHelper
return certs; return certs;
} }
/// <summary>
/// Builds server TLS authentication options from configured NATS TLS settings.
/// </summary>
/// <param name="opts">Server options containing TLS and OCSP settings.</param>
/// <returns>Configured SSL server authentication options.</returns>
public static SslServerAuthenticationOptions BuildServerAuthOptions(NatsOptions opts) public static SslServerAuthenticationOptions BuildServerAuthOptions(NatsOptions opts)
{ {
var cert = LoadCertificate(opts.TlsCert!, opts.TlsKey); var cert = LoadCertificate(opts.TlsCert!, opts.TlsKey);
@@ -117,6 +135,9 @@ public static class TlsHelper
/// When <paramref name="offline"/> is false the runtime will contact the /// When <paramref name="offline"/> is false the runtime will contact the
/// certificate's OCSP responder to obtain a fresh stapled response. /// certificate's OCSP responder to obtain a fresh stapled response.
/// </summary> /// </summary>
/// <param name="opts">Server options containing TLS and OCSP settings.</param>
/// <param name="offline">When <see langword="true"/>, avoids online OCSP fetch while building context.</param>
/// <returns>Certificate context for SSL stream usage, or <see langword="null"/>.</returns>
public static SslStreamCertificateContext? BuildCertificateContext(NatsOptions opts, bool offline = false) public static SslStreamCertificateContext? BuildCertificateContext(NatsOptions opts, bool offline = false)
{ {
if (!opts.HasTls) return null; if (!opts.HasTls) return null;
@@ -130,6 +151,11 @@ public static class TlsHelper
return SslStreamCertificateContext.Create(cert, chain, offline: offline); return SslStreamCertificateContext.Create(cert, chain, offline: offline);
} }
/// <summary>
/// Computes the SHA-256 hash of certificate SubjectPublicKeyInfo bytes.
/// </summary>
/// <param name="cert">Certificate to hash.</param>
/// <returns>Lowercase hexadecimal hash string.</returns>
public static string GetCertificateHash(X509Certificate2 cert) public static string GetCertificateHash(X509Certificate2 cert)
{ {
var spki = cert.PublicKey.ExportSubjectPublicKeyInfo(); var spki = cert.PublicKey.ExportSubjectPublicKeyInfo();
@@ -137,12 +163,22 @@ public static class TlsHelper
return Convert.ToHexStringLower(hash); return Convert.ToHexStringLower(hash);
} }
/// <summary>
/// Generates a base64-encoded SHA-256 fingerprint for the raw certificate.
/// </summary>
/// <param name="cert">Certificate to fingerprint.</param>
/// <returns>Base64 fingerprint string.</returns>
public static string GenerateFingerprint(X509Certificate2 cert) public static string GenerateFingerprint(X509Certificate2 cert)
{ {
var hash = SHA256.HashData(cert.RawData); var hash = SHA256.HashData(cert.RawData);
return Convert.ToBase64String(hash); return Convert.ToBase64String(hash);
} }
/// <summary>
/// Filters a URI set to valid absolute HTTP(S) endpoints.
/// </summary>
/// <param name="uris">Candidate URI strings.</param>
/// <returns>Validated HTTP(S) endpoint list.</returns>
public static IReadOnlyList<Uri> GetWebEndpoints(IEnumerable<string> uris) public static IReadOnlyList<Uri> GetWebEndpoints(IEnumerable<string> uris)
{ {
var urls = new List<Uri>(); var urls = new List<Uri>();
@@ -159,16 +195,32 @@ public static class TlsHelper
return urls; return urls;
} }
/// <summary>
/// Returns certificate subject distinguished-name text.
/// </summary>
/// <param name="cert">Certificate to inspect.</param>
/// <returns>Subject DN string, or empty when unavailable.</returns>
public static string GetSubjectDNForm(X509Certificate2? cert) public static string GetSubjectDNForm(X509Certificate2? cert)
{ {
return cert?.SubjectName.Name ?? string.Empty; return cert?.SubjectName.Name ?? string.Empty;
} }
/// <summary>
/// Returns certificate issuer distinguished-name text.
/// </summary>
/// <param name="cert">Certificate to inspect.</param>
/// <returns>Issuer DN string, or empty when unavailable.</returns>
public static string GetIssuerDNForm(X509Certificate2? cert) public static string GetIssuerDNForm(X509Certificate2? cert)
{ {
return cert?.IssuerName.Name ?? string.Empty; return cert?.IssuerName.Name ?? string.Empty;
} }
/// <summary>
/// Checks whether a certificate matches the configured pinned-cert set.
/// </summary>
/// <param name="cert">Certificate to test.</param>
/// <param name="pinned">Pinned certificate hash set.</param>
/// <returns><see langword="true"/> when certificate hash is pinned.</returns>
public static bool MatchesPinnedCert(X509Certificate2 cert, HashSet<string> pinned) public static bool MatchesPinnedCert(X509Certificate2 cert, HashSet<string> pinned)
{ {
var hash = GetCertificateHash(cert); var hash = GetCertificateHash(cert);
@@ -179,6 +231,8 @@ public static class TlsHelper
/// Checks if a chain link is eligible for OCSP validation by ensuring the leaf /// Checks if a chain link is eligible for OCSP validation by ensuring the leaf
/// certificate includes at least one valid HTTP(S) OCSP AIA endpoint. /// certificate includes at least one valid HTTP(S) OCSP AIA endpoint.
/// </summary> /// </summary>
/// <param name="link">Chain link containing leaf certificate metadata.</param>
/// <returns><see langword="true"/> when OCSP checking can be performed.</returns>
public static bool CertOCSPEligible(ChainLink? link) public static bool CertOCSPEligible(ChainLink? link)
{ {
if (link?.Leaf is null) if (link?.Leaf is null)
@@ -202,6 +256,9 @@ public static class TlsHelper
/// <summary> /// <summary>
/// Returns the positional issuer certificate for a leaf in a verified chain. /// Returns the positional issuer certificate for a leaf in a verified chain.
/// </summary> /// </summary>
/// <param name="chain">Verified certificate chain.</param>
/// <param name="leafPos">Index of leaf certificate in chain.</param>
/// <returns>Issuer certificate, or <see langword="null"/> when unavailable.</returns>
public static X509Certificate2? GetLeafIssuerCert(IReadOnlyList<X509Certificate2>? chain, int leafPos) public static X509Certificate2? GetLeafIssuerCert(IReadOnlyList<X509Certificate2>? chain, int leafPos)
{ {
if (chain is null || chain.Count == 0 || leafPos < 0 || leafPos >= chain.Count - 1) if (chain is null || chain.Count == 0 || leafPos < 0 || leafPos >= chain.Count - 1)
@@ -213,6 +270,9 @@ public static class TlsHelper
/// Equivalent to Go certstore.GetLeafIssuer: verifies the leaf against the /// Equivalent to Go certstore.GetLeafIssuer: verifies the leaf against the
/// supplied trust root and returns the first issuer in the verified chain. /// supplied trust root and returns the first issuer in the verified chain.
/// </summary> /// </summary>
/// <param name="leaf">Leaf certificate.</param>
/// <param name="trustedRoot">Trusted root certificate.</param>
/// <returns>Issuer certificate from built chain, or <see langword="null"/>.</returns>
public static X509Certificate2? GetLeafIssuer(X509Certificate2 leaf, X509Certificate2 trustedRoot) public static X509Certificate2? GetLeafIssuer(X509Certificate2 leaf, X509Certificate2 trustedRoot)
{ {
using var chain = new X509Chain(); using var chain = new X509Chain();
@@ -230,6 +290,9 @@ public static class TlsHelper
/// <summary> /// <summary>
/// Checks OCSP response currency semantics with clock skew and fallback TTL. /// Checks OCSP response currency semantics with clock skew and fallback TTL.
/// </summary> /// </summary>
/// <param name="response">Parsed OCSP response timestamps.</param>
/// <param name="opts">OCSP peer-validation options.</param>
/// <returns><see langword="true"/> when response timestamps are currently valid.</returns>
public static bool OcspResponseCurrent(OcspResponseInfo response, OCSPPeerConfig opts) public static bool OcspResponseCurrent(OcspResponseInfo response, OCSPPeerConfig opts)
{ {
var skew = TimeSpan.FromSeconds(opts.ClockSkew); var skew = TimeSpan.FromSeconds(opts.ClockSkew);
@@ -261,6 +324,9 @@ public static class TlsHelper
/// Validates OCSP delegated signer semantics. Direct issuer signatures are valid; /// Validates OCSP delegated signer semantics. Direct issuer signatures are valid;
/// delegated certificates must include id-kp-OCSPSigning EKU. /// delegated certificates must include id-kp-OCSPSigning EKU.
/// </summary> /// </summary>
/// <param name="issuer">Issuer certificate.</param>
/// <param name="responderCertificate">Optional delegated OCSP responder certificate.</param>
/// <returns><see langword="true"/> when delegation/signing semantics are valid.</returns>
public static bool ValidDelegationCheck(X509Certificate2? issuer, X509Certificate2? responderCertificate) public static bool ValidDelegationCheck(X509Certificate2? issuer, X509Certificate2? responderCertificate)
{ {
if (issuer is null) if (issuer is null)
+28
View File
@@ -20,9 +20,20 @@ public sealed class WsConnection : Stream
private readonly object _writeLock = new(); private readonly object _writeLock = new();
private readonly List<ControlFrameAction> _pendingControlWrites = []; private readonly List<ControlFrameAction> _pendingControlWrites = [];
/// <summary>Indicates whether a close frame has been received from the remote peer.</summary>
public bool CloseReceived => _readInfo.CloseReceived; public bool CloseReceived => _readInfo.CloseReceived;
/// <summary>WebSocket close status code received from the remote peer.</summary>
public int CloseStatus => _readInfo.CloseStatus; public int CloseStatus => _readInfo.CloseStatus;
/// <summary>
/// Initializes a WebSocket framed stream wrapper over an existing transport stream.
/// </summary>
/// <param name="inner">Underlying network stream.</param>
/// <param name="compress">Whether outbound frames may use per-message compression.</param>
/// <param name="maskRead">Whether inbound frames are expected to be masked.</param>
/// <param name="maskWrite">Whether outbound frames should be masked.</param>
/// <param name="browser">Whether browser framing constraints should be applied.</param>
/// <param name="noCompFrag">Whether compressed payloads should avoid browser fragmentation logic.</param>
public WsConnection(Stream inner, bool compress, bool maskRead, bool maskWrite, bool browser, bool noCompFrag) public WsConnection(Stream inner, bool compress, bool maskRead, bool maskWrite, bool browser, bool noCompFrag)
{ {
_inner = inner; _inner = inner;
@@ -34,6 +45,7 @@ public sealed class WsConnection : Stream
_readInfo = new WsReadInfo(expectMask: maskRead); _readInfo = new WsReadInfo(expectMask: maskRead);
} }
/// <inheritdoc />
public override async ValueTask<int> ReadAsync(Memory<byte> buffer, CancellationToken ct = default) public override async ValueTask<int> ReadAsync(Memory<byte> buffer, CancellationToken ct = default)
{ {
// Drain any buffered decoded payloads first // Drain any buffered decoded payloads first
@@ -73,6 +85,7 @@ public sealed class WsConnection : Stream
} }
} }
/// <inheritdoc />
public override async ValueTask WriteAsync(ReadOnlyMemory<byte> buffer, CancellationToken ct = default) public override async ValueTask WriteAsync(ReadOnlyMemory<byte> buffer, CancellationToken ct = default)
{ {
var data = buffer.Span; var data = buffer.Span;
@@ -145,6 +158,8 @@ public sealed class WsConnection : Stream
/// <summary> /// <summary>
/// Sends a WebSocket close frame. /// Sends a WebSocket close frame.
/// </summary> /// </summary>
/// <param name="reason">Server close reason mapped to WebSocket close status and reason text.</param>
/// <param name="ct">Cancellation token for asynchronous close frame transmission.</param>
public async Task SendCloseAsync(ClientClosedReason reason, CancellationToken ct = default) public async Task SendCloseAsync(ClientClosedReason reason, CancellationToken ct = default)
{ {
var status = WsFrameWriter.MapCloseStatus(reason); var status = WsFrameWriter.MapCloseStatus(reason);
@@ -175,18 +190,30 @@ public sealed class WsConnection : Stream
} }
// Stream abstract members // Stream abstract members
/// <inheritdoc />
public override bool CanRead => true; public override bool CanRead => true;
/// <inheritdoc />
public override bool CanWrite => true; public override bool CanWrite => true;
/// <inheritdoc />
public override bool CanSeek => false; public override bool CanSeek => false;
/// <inheritdoc />
public override long Length => throw new NotSupportedException(); public override long Length => throw new NotSupportedException();
/// <inheritdoc />
public override long Position { get => throw new NotSupportedException(); set => throw new NotSupportedException(); } public override long Position { get => throw new NotSupportedException(); set => throw new NotSupportedException(); }
/// <inheritdoc />
public override void Flush() => _inner.Flush(); public override void Flush() => _inner.Flush();
/// <inheritdoc />
public override Task FlushAsync(CancellationToken ct) => _inner.FlushAsync(ct); public override Task FlushAsync(CancellationToken ct) => _inner.FlushAsync(ct);
/// <inheritdoc />
public override int Read(byte[] buffer, int offset, int count) => throw new NotSupportedException("Use ReadAsync"); public override int Read(byte[] buffer, int offset, int count) => throw new NotSupportedException("Use ReadAsync");
/// <inheritdoc />
public override void Write(byte[] buffer, int offset, int count) => throw new NotSupportedException("Use WriteAsync"); public override void Write(byte[] buffer, int offset, int count) => throw new NotSupportedException("Use WriteAsync");
/// <inheritdoc />
public override long Seek(long offset, SeekOrigin origin) => throw new NotSupportedException(); public override long Seek(long offset, SeekOrigin origin) => throw new NotSupportedException();
/// <inheritdoc />
public override void SetLength(long value) => throw new NotSupportedException(); public override void SetLength(long value) => throw new NotSupportedException();
/// <inheritdoc />
protected override void Dispose(bool disposing) protected override void Dispose(bool disposing)
{ {
if (disposing) if (disposing)
@@ -194,6 +221,7 @@ public sealed class WsConnection : Stream
base.Dispose(disposing); base.Dispose(disposing);
} }
/// <inheritdoc />
public override async ValueTask DisposeAsync() public override async ValueTask DisposeAsync()
{ {
await _inner.DisposeAsync(); await _inner.DisposeAsync();
@@ -14,6 +14,11 @@ public static class WsFrameWriter
/// Creates a complete frame header for a single-frame message (first=true, final=true). /// Creates a complete frame header for a single-frame message (first=true, final=true).
/// Returns (header bytes, mask key or null). /// Returns (header bytes, mask key or null).
/// </summary> /// </summary>
/// <param name="useMasking">Whether to include a masking key and mask payload bytes.</param>
/// <param name="compressed">Whether to set RSV1 for permessage-deflate.</param>
/// <param name="opcode">WebSocket opcode for the frame.</param>
/// <param name="payloadLength">Payload length in bytes.</param>
/// <returns>Encoded header bytes and optional mask key.</returns>
public static (byte[] header, byte[]? key) CreateFrameHeader( public static (byte[] header, byte[]? key) CreateFrameHeader(
bool useMasking, bool compressed, int opcode, int payloadLength) bool useMasking, bool compressed, int opcode, int payloadLength)
{ {
@@ -27,6 +32,14 @@ public static class WsFrameWriter
/// Fills a pre-allocated frame header buffer. /// Fills a pre-allocated frame header buffer.
/// Returns (bytes written, mask key or null). /// Returns (bytes written, mask key or null).
/// </summary> /// </summary>
/// <param name="fh">Destination buffer for header bytes.</param>
/// <param name="useMasking">Whether to include masking metadata.</param>
/// <param name="first">Whether this is the first frame in a message.</param>
/// <param name="final">Whether this is the final frame in a message.</param>
/// <param name="compressed">Whether to set RSV1 for permessage-deflate.</param>
/// <param name="opcode">WebSocket opcode for the frame.</param>
/// <param name="payloadLength">Payload length in bytes.</param>
/// <returns>Number of bytes written and optional mask key.</returns>
public static (int written, byte[]? key) FillFrameHeader( public static (int written, byte[]? key) FillFrameHeader(
Span<byte> fh, bool useMasking, bool first, bool final, bool compressed, int opcode, int payloadLength) Span<byte> fh, bool useMasking, bool first, bool final, bool compressed, int opcode, int payloadLength)
{ {
@@ -74,6 +87,8 @@ public static class WsFrameWriter
/// <summary> /// <summary>
/// XOR masks a buffer with a 4-byte key. Applies in-place. /// XOR masks a buffer with a 4-byte key. Applies in-place.
/// </summary> /// </summary>
/// <param name="key">Four-byte masking key.</param>
/// <param name="buf">Buffer to mask in place.</param>
public static void MaskBuf(ReadOnlySpan<byte> key, Span<byte> buf) public static void MaskBuf(ReadOnlySpan<byte> key, Span<byte> buf)
{ {
for (int i = 0; i < buf.Length; i++) for (int i = 0; i < buf.Length; i++)
@@ -83,6 +98,8 @@ public static class WsFrameWriter
/// <summary> /// <summary>
/// XOR masks multiple contiguous buffers as if they were one. /// XOR masks multiple contiguous buffers as if they were one.
/// </summary> /// </summary>
/// <param name="key">Four-byte masking key.</param>
/// <param name="bufs">Buffers to mask as a contiguous payload sequence.</param>
public static void MaskBufs(ReadOnlySpan<byte> key, List<byte[]> bufs) public static void MaskBufs(ReadOnlySpan<byte> key, List<byte[]> bufs)
{ {
int pos = 0; int pos = 0;
@@ -100,6 +117,9 @@ public static class WsFrameWriter
/// Creates a close message payload: 2-byte status code + optional UTF-8 body. /// Creates a close message payload: 2-byte status code + optional UTF-8 body.
/// Body truncated to fit MaxControlPayloadSize with "..." suffix. /// Body truncated to fit MaxControlPayloadSize with "..." suffix.
/// </summary> /// </summary>
/// <param name="status">WebSocket close status code.</param>
/// <param name="body">Optional close reason text.</param>
/// <returns>Encoded close payload bytes.</returns>
public static byte[] CreateCloseMessage(int status, string body) public static byte[] CreateCloseMessage(int status, string body)
{ {
var bodyBytes = Encoding.UTF8.GetBytes(body); var bodyBytes = Encoding.UTF8.GetBytes(body);
@@ -128,6 +148,10 @@ public static class WsFrameWriter
/// <summary> /// <summary>
/// Builds a complete control frame (header + payload, optional masking). /// Builds a complete control frame (header + payload, optional masking).
/// </summary> /// </summary>
/// <param name="opcode">Control frame opcode.</param>
/// <param name="payload">Control frame payload bytes.</param>
/// <param name="useMasking">Whether to apply client-style masking.</param>
/// <returns>Complete encoded control frame.</returns>
public static byte[] BuildControlFrame(int opcode, ReadOnlySpan<byte> payload, bool useMasking) public static byte[] BuildControlFrame(int opcode, ReadOnlySpan<byte> payload, bool useMasking)
{ {
int headerSize = 2 + (useMasking ? 4 : 0); int headerSize = 2 + (useMasking ? 4 : 0);
@@ -149,6 +173,8 @@ public static class WsFrameWriter
/// Maps a ClientClosedReason to a WebSocket close status code. /// Maps a ClientClosedReason to a WebSocket close status code.
/// Matches Go wsEnqueueCloseMessage in websocket.go lines 668-694. /// Matches Go wsEnqueueCloseMessage in websocket.go lines 668-694.
/// </summary> /// </summary>
/// <param name="reason">Connection close reason.</param>
/// <returns>WebSocket close status code.</returns>
public static int MapCloseStatus(ClientClosedReason reason) => reason switch public static int MapCloseStatus(ClientClosedReason reason) => reason switch
{ {
ClientClosedReason.ClientClosed => WsConstants.CloseStatusNormalClosure, ClientClosedReason.ClientClosed => WsConstants.CloseStatusNormalClosure,
@@ -16,7 +16,7 @@ public class OrderedConsumerTests(JetStreamServerPairFixture fixture, ITestOutpu
public async Task JSOrderedConsumer_Throughput() public async Task JSOrderedConsumer_Throughput()
{ {
const int payloadSize = 128; const int payloadSize = 128;
const int messageCount = 25_000; const int messageCount = 200_000;
BenchmarkResult? dotnetResult = null; BenchmarkResult? dotnetResult = null;
try try