Files
natsdotnet/differences.md
Joseph Doherty 3531a87de0 Merge branch 'feature/system-account-types'
Add SYSTEM and ACCOUNT connection types with InternalClient,
InternalEventSystem, system event publishing, request-reply services,
and cross-account import/export support.
2026-02-23 06:14:11 -05:00

22 KiB

Go vs .NET NATS Server: Functionality Differences

Excludes clustering/routes, gateways, leaf nodes, and JetStream. Generated 2026-02-22 by comparing golang/nats-server/server/ against src/NATS.Server/.


1. Core Server Lifecycle

Server Initialization

Feature Go .NET Notes
NKey generation (server identity) Y Y Ed25519 key pair via NATS.NKeys at startup
System account setup Y Y $SYS account with InternalEventSystem, event publishing, request-reply services
Config file validation on startup Y Y Full config parsing with error collection via ConfigProcessor
PID file writing Y Y Written on startup, deleted on shutdown
Profiling HTTP endpoint (/debug/pprof) Y Stub ProfPort option exists but endpoint not implemented
Ports file output Y Y JSON ports file written to PortsFileDir on startup

Accept Loop

Feature Go .NET Notes
Exponential backoff on accept errors Y Y .NET backs off from 10ms to 1s on repeated failures
Config reload lock during client creation Y N Go holds reloadMu around createClient
Goroutine/task tracking (WaitGroup) Y Y Interlocked counter + drain with 10s timeout on shutdown
Callback-based error handling Y N Go uses errFunc callback pattern
Random/ephemeral port (port=0) Y Y Port resolved after Bind+Listen, stored in _options.Port

Shutdown

Feature Go .NET Notes
Graceful shutdown with WaitForShutdown() Y Y Idempotent CAS-guarded ShutdownAsync() + blocking WaitForShutdown()
Close reason tracking per connection Y Y 37-value ClosedState enum, CAS-based first-writer-wins MarkClosed()
Lame duck mode (stop new, drain existing) Y Y LameDuckShutdownAsync() with grace period + stagger-close with jitter
Wait for accept loop completion Y Y TaskCompletionSource signaled in accept loop finally
Flush pending data before close Y Y FlushAndCloseAsync() with best-effort flush, skip-flush for error conditions

Signal Handling

Signal Go .NET Notes
SIGINT (Ctrl+C) Y Y Both handle graceful shutdown
SIGTERM Y Y PosixSignalRegistration triggers ShutdownAsync()
SIGUSR1 (reopen logs) Y Y SIGUSR1 handler calls ReOpenLogFile
SIGUSR2 (lame duck mode) Y Y Triggers LameDuckShutdownAsync()
SIGHUP (config reload) Y Y Re-parses config, diffs options, applies reloadable subset; CLI flags preserved
Windows Service integration Y Y --service flag with Microsoft.Extensions.Hosting.WindowsServices

2. Client / Connection Handling

Concurrency Model

Feature Go .NET Notes
Separate read + write loops Y Y Channel-based RunWriteLoopAsync with QueueOutbound()
Write coalescing / batch flush Y Y Write loop drains all channel items before single FlushAsync
Dynamic buffer sizing (512B-64KB) Y N .NET delegates to System.IO.Pipelines
Output buffer pooling (3-tier) Y N Go pools at 512B, 4KB, 64KB

Connection Types

Type Go .NET Notes
CLIENT Y Y
ROUTER Y N Excluded per scope
GATEWAY Y N Excluded per scope
LEAF Y N Excluded per scope
SYSTEM (internal) Y Y InternalClient + InternalEventSystem with Channel-based send/receive loops
JETSTREAM (internal) Y N
ACCOUNT (internal) Y Y Lazy per-account InternalClient with import/export subscription support
WebSocket clients Y Y Custom frame parser, permessage-deflate compression, origin checking, cookie auth
MQTT clients Y N

Client Features

Feature Go .NET Notes
Echo suppression (echo: false) Y Y .NET checks echo in delivery path (NatsServer.cs:234,253)
Verbose mode (+OK responses) Y Y Sends +OK after CONNECT, SUB, UNSUB, PUB when verbose:true
No-responders validation Y Y CONNECT rejects no_responders without headers; 503 HMSG on no match
Slow consumer detection Y Y Pending bytes threshold (64MB) + write deadline timeout (10s)
Write deadline / timeout policies Y Y WriteDeadline option with CancellationTokenSource.CancelAfter on flush
RTT measurement Y Y _rttStartTicks/Rtt property, computed on PONG receipt
Per-client trace mode Y Y SetTraceMode() toggles parser logger dynamically via ClientFlags.TraceMode
Detailed close reason tracking Y Y 37-value ClosedState enum with CAS-based MarkClosed()
Connection state flags (16 flags) Y Y 7-flag ClientFlagHolder with Interlocked.Or/And

Slow Consumer Handling

Go implements a sophisticated slow consumer detection system:

  • Tracks pendingBytes per client output buffer
  • If pending exceeds maxPending, enters stall mode (2-5ms waits)
  • Total stall capped at 10ms per read cycle
  • Closes with SlowConsumerPendingBytes or SlowConsumerWriteDeadline
  • Sets isSlowConsumer flag for monitoring

.NET now implements pending bytes tracking and write deadline enforcement via Channel<ReadOnlyMemory<byte>>. Key differences from Go: no stall/retry mode (immediate close on threshold exceeded), write deadline via CancellationTokenSource.CancelAfter instead of SetWriteDeadline. IsSlowConsumer flag and server-level SlowConsumerCount stats are tracked for monitoring.

Stats Tracking

Feature Go .NET Notes
Per-connection atomic stats Y Y .NET uses Interlocked for stats access
Per-read-cycle stat batching Y Y Local accumulators flushed via Interlocked.Add per read cycle
Per-account stats Y Y Interlocked counters for InMsgs/OutMsgs/InBytes/OutBytes per Account
Slow consumer counters Y Y SlowConsumers and SlowConsumerClients incremented on detection

3. Protocol Parsing

Parser Architecture

Aspect Go .NET
Approach Byte-by-byte state machine (74 states) Two-phase: line extraction + command dispatch
Case handling Per-state character checks Bit-mask lowercase normalization (| 0x20)
Buffer strategy Jump-ahead optimization for payloads Direct size-based reads via Pipe
Split-buffer handling argBuf accumulation with scratch buffer State variables (_awaitingPayload, etc.)
Error model Inline error sending + error return Exception-based (ProtocolViolationException)
CRLF in payload Included in message buffer Excluded by design

Protocol Operations

Operation Go .NET Notes
PUB Y Y
HPUB (headers) Y Y
SUB Y Y
UNSUB Y Y
CONNECT Y Y
INFO Y Y
PING / PONG Y Y
MSG / HMSG Y Y
+OK / -ERR Y Y
RS+/RS-/RMSG (routes) Y N Excluded per scope
A+/A- (accounts) Y N Excluded per scope
LS+/LS-/LMSG (leaf) Y N Excluded per scope

Protocol Parsing Gaps

Feature Go .NET Notes
Multi-client-type command routing Y N Go checks c.kind to allow/reject commands
Protocol tracing in parser Y Y TraceInOp() logs <<- OP arg at LogLevel.Trace via optional ILogger
Subject mapping (input→output) Y Y Compiled SubjectTransform engine with 9 function tokens; wired into ProcessMessage
MIME header parsing Y Y NatsHeaderParser.Parse() — status line + key-value headers from ReadOnlySpan<byte>
Message trace event initialization Y N

Protocol Writing

Aspect Go .NET Notes
INFO serialization Once at startup Once at startup Cached at startup; nonce connections serialize per-connection
MSG/HMSG construction Direct buffer write Span-based buffer write int.TryFormat + CopyTo into rented buffer, no string allocations
Pre-encoded constants Y Y Both pre-encode PING/PONG/OK

4. Subscriptions & Subject Matching

Trie Implementation

Feature Go .NET Notes
Basic trie with */> wildcards Y Y Core matching identical
Queue group support Y Y
Result caching (1024 max) Y Y Same limits
plist optimization (>256 subs) Y N Go converts high-fanout nodes to array
Async cache sweep (background) Y N .NET sweeps inline under write lock
Atomic generation ID for invalidation Y Y Interlocked.Increment on insert/remove; cached results store generation
Cache eviction strategy Random First-N Semantic difference minimal

SubList Features

Feature Go .NET Notes
Stats() — comprehensive statistics Y Y Matches, cache hits, inserts, removes tracked via Interlocked
HasInterest() — fast bool check Y Y Walks trie without allocating result list
NumInterest() — fast count Y Y Counts plain + queue subs without allocation
ReverseMatch() — pattern→literal query Y Y Finds subscriptions whose wildcards match a literal subject
RemoveBatch() — efficient bulk removal Y Y Single generation increment for batch; increments _removes per sub
All() — enumerate all subscriptions Y Y Recursive trie walk returning all subscriptions
Notification system (interest changes) Y N
Local/remote subscription filtering Y N
Queue weight expansion (remote subs) Y N
MatchBytes() — zero-copy byte API Y N

Subject Validation

Feature Go .NET Notes
Basic validation (empty tokens, wildcards) Y Y
Literal subject check Y Y
UTF-8/null rune validation Y Y IsValidSubject(string, bool checkRunes) rejects null bytes
Collision detection (SubjectsCollide) Y Y Token-by-token wildcard comparison; O(n) via upfront Split
Token utilities (tokenAt, numTokens) Y Y TokenAt returns ReadOnlySpan<char>; NumTokens counts separators
Stack-allocated token buffer Y N Go uses [32]string{} on stack

Subscription Lifecycle

Feature Go .NET Notes
Per-account subscription limit Y Y Account.IncrementSubscriptions() returns false when MaxSubscriptions exceeded
Auto-unsubscribe on max messages Y Y Enforced at delivery; sub removed from trie + client dict when exhausted
Subscription routing propagation Y N For clusters
Queue weight (qw) field Y N For remote queue load balancing

5. Authentication & Authorization

Auth Mechanisms

Mechanism Go .NET Notes
Username/password Y Y
Token Y Y
NKeys (Ed25519) Y Y .NET has framework but integration is basic
JWT validation Y Y NatsJwt decode/verify, JwtAuthenticator with account resolution + revocation
Bcrypt password hashing Y Y .NET supports bcrypt ($2* prefix) with constant-time fallback
TLS certificate mapping Y Y X500DistinguishedName with full DN match and CN fallback
Custom auth interface Y N
External auth callout Y N
Proxy authentication Y N
Bearer tokens Y Y UserClaims.BearerToken skips nonce signature verification
User revocation tracking Y Y Per-account ConcurrentDictionary with wildcard (*) revocation support

Account System

Feature Go .NET Notes
Per-account SubList isolation Y Y
Multi-account user resolution Y Y AccountConfig per account in NatsOptions.Accounts; GetOrCreateAccount wires limits
Account exports/imports Y Y ServiceImport/StreamImport with ExportAuth, subject transforms, response routing
Per-account connection limits Y Y Account.AddClient() returns false when MaxConnections exceeded
Per-account subscription limits Y Y Account.IncrementSubscriptions() enforced in ProcessSub()
Account JetStream limits Y N Excluded per scope

Permissions

Feature Go .NET Notes
Publish allow list Y Y
Subscribe allow list Y Y
Publish deny list Y Y Full enforcement with LRU-cached results
Subscribe deny list Y Y Queue-aware deny checking in IsSubscribeAllowed
Message-level deny filtering Y Y IsDeliveryAllowed() checked before MSG send; auto-unsub cleanup on deny
Permission caching (128 entries) Y Y PermissionLruCache — Dictionary+LinkedList LRU, matching Go's maxPermCacheSize
Response permissions (reply tracking) Y Y ResponseTracker with configurable TTL + max messages; not LRU-cached
Auth expiry enforcement Y Y Task.Delay timer closes client when JWT/auth expires
Permission templates (JWT) Y Y PermissionTemplates.Expand() — 6 functions with cartesian product for multi-value tags

6. Configuration

CLI Flags

Flag Go .NET Notes
-p/--port Y Y
-a/--addr Y Y
-n/--name (ServerName) Y Y
-m/--http_port (monitoring) Y Y
-c (config file) Y Y Full config parsing: lexer → parser → processor; CLI args override config
-D/-V/-DV (debug/trace) Y Y -D/--debug for debug, -V/-T/--trace for trace, -DV for both
--tlscert/--tlskey/--tlscacert Y Y
--tlsverify Y Y
--http_base_path Y Y
--https_port Y Y

Configuration System

Feature Go .NET Notes
Config file parsing Y Y Custom NATS conf lexer/parser ported from Go; supports includes, variables, blocks
Hot reload (SIGHUP) Y Y Reloads logging, auth, limits, TLS certs on SIGHUP; rejects non-reloadable changes
Config change detection Y Y SHA256 digest comparison; InCmdLine tracks CLI flag precedence
~450 option fields Y ~72 .NET covers core + all single-server options; cluster/JetStream keys silently ignored

Missing Options Categories

  • Logging options — file logging, rotation, syslog, debug/trace, color, timestamps, per-subsystem log control all implemented
  • Advanced limits (MaxSubs, MaxSubTokens, MaxPending, WriteDeadline)MaxSubs, MaxSubTokens implemented; MaxPending/WriteDeadline already existed
  • Tags/metadataTags dictionary implemented in NatsOptions
  • OCSP configurationOcspConfig with 4 modes (Auto/Always/Must/Never), peer verification, and stapling
  • WebSocket optionsWebSocketOptions with port, compression, origin checking, cookie auth, custom headers
  • MQTT options
  • Operator mode / account resolverJwtAuthenticator + IAccountResolver + MemAccountResolver with trusted keys

7. Monitoring

HTTP Endpoints

Endpoint Go .NET Notes
/healthz Y Y
/varz Y Y
/connz Y Y
/ (root listing) Y Y
/routez Y Stub Returns empty response
/gatewayz Y Stub Returns empty response
/leafz Y Stub Returns empty response
/subz / /subscriptionsz Y Y Account filtering, test subject filtering, pagination, and subscription details
/accountz Y Stub Returns empty response
/accstatz Y Stub Returns empty response
/jsz Y Stub Returns empty response

Varz Response

Field Category Go .NET Notes
Identity (ID, Name, Version) Y Y
Network (Host, Port, URLs) Y Y
Security (AuthRequired, TLS) Y Y
Limits (MaxConn, MaxPayload) Y Y
Timing (Start, Now, Uptime) Y Y
Runtime (Mem, CPU, Cores) Y Y
Connections (current, total) Y Y
Messages (in/out msgs/bytes) Y Y
SlowConsumer breakdown Y N Go tracks per connection type
Cluster/Gateway/Leaf blocks Y N Excluded per scope
JetStream block Y N Excluded per scope
TLS cert expiry info Y Y TlsCertNotAfter loaded via X509CertificateLoader in /varz

Connz Response

Feature Go .NET Notes
Filtering by CID, user, account Y Partial
Sorting (11 options) Y Y All options including ByStop, ByReason, ByRtt
State filtering (open/closed/all) Y Y `state=open
Closed connection tracking Y Y ConcurrentQueue<ClosedClient> capped at 10,000 entries
Pagination (offset, limit) Y Y
Subscription detail mode Y N
TLS peer certificate info Y N
JWT/IssuerKey/Tags fields Y N
MQTT client ID filtering Y N
Proxy info Y N

8. TLS

TLS Modes

Mode Go .NET Notes
No TLS Y Y
INFO-first (default NATS) Y Y
TLS-first (before INFO) Y Y
Mixed/Fallback Y Y
TLS-required Y Y

TLS Features

Feature Go .NET Notes
PEM cert/key loading Y Y
CA chain validation Y Y
Mutual TLS (client certs) Y Y
Certificate pinning (SHA256 SPKI) Y Y
TLS handshake timeout Y Y
TLS rate limiting Y Y Rate enforcement with refill; unit tests cover rate limiting and refill
First-byte peeking (0x16 detection) Y Y
Cert subject→user mapping Y Y X500DistinguishedName with full DN match and CN fallback
OCSP stapling Y Y SslStreamCertificateContext.Create with offline:false for runtime OCSP fetch
Min TLS version control Y Y

9. Logging

Feature Go .NET Notes
Structured logging Partial Y .NET uses Serilog with ILogger
File logging with rotation Y Y -l/--log_file flag + LogSizeLimit/LogMaxFiles via Serilog.Sinks.File
Syslog (local and remote) Y Y --syslog and --remote_syslog flags via Serilog.Sinks.SyslogMessages
Log reopening (SIGUSR1) Y Y SIGUSR1 handler calls ReOpenLogFile callback
Trace mode (protocol-level) Y Y -V/-T/--trace flags; parser TraceInOp() logs at Trace level
Debug mode Y Y -D/--debug flag lowers Serilog minimum to Debug
Per-subsystem log control Y Y --log_level_override ns=level CLI flag with Serilog MinimumLevel.Override
Color output on TTY Y Y Auto-detected via Console.IsOutputRedirected, uses AnsiConsoleTheme.Code
Timestamp format control Y Y --logtime and --logtime_utc flags

10. Ping/Pong & Keepalive

Feature Go .NET Notes
Server-initiated PING Y Y
Configurable interval Y Y PingInterval option
Max pings out Y Y MaxPingsOut option
Stale connection close Y Y
RTT-based first PING delay Y Y Skips PING until FirstPongSent or 2s elapsed
RTT tracking Y Y _rttStartTicks/Rtt property, computed on PONG receipt
Stale connection stats Y Y StaleConnectionStats model, exposed in /varz

Summary: Critical Gaps for Production Use

Resolved Since Initial Audit

The following items from the original gap list have been implemented:

  • Slow consumer detection — pending bytes threshold (64MB) with write deadline enforcement
  • Write coalescing / batch flush — channel-based write loop drains all items before single flush
  • Verbose mode+OK responses for CONNECT, SUB, UNSUB, PUB when verbose:true
  • Permission deny enforcement at deliveryIsDeliveryAllowed + auto-unsub cleanup
  • No-responders validation — CONNECT rejects no_responders without headers; 503 HMSG on no match
  • File logging with rotation — Serilog.Sinks.File with rolling file support
  • TLS certificate mapping — X500DistinguishedName with full DN match and CN fallback
  • Protocol tracing-V/-T flag enables trace-level logging; -D for debug
  • Subscription statisticsStats(), HasInterest(), NumInterest(), etc.
  • Per-account limits — connection + subscription limits via AccountConfig
  • Reply subject trackingResponseTracker with TTL + max messages
  • JWT authenticationJwtAuthenticator with decode/verify, account resolution, revocation, permission templates
  • OCSP support — peer verification via X509RevocationMode.Online, stapling via SslStreamCertificateContext
  • Subject mapping — compiled SubjectTransform engine with 9 function tokens, wired into message delivery
  • Windows Service integration--service flag with Microsoft.Extensions.Hosting.WindowsServices
  • Per-subsystem log control--log_level_override CLI flag with Serilog overrides
  • Per-client trace modeSetTraceMode() with dynamic parser logger toggling
  • Per-account statsInterlocked counters for InMsgs/OutMsgs/InBytes/OutBytes
  • TLS cert expiry in /varzTlsCertNotAfter populated via X509CertificateLoader
  • Permission templatesPermissionTemplates.Expand() with 6 functions and cartesian product
  • Bearer tokensUserClaims.BearerToken skips nonce verification
  • User revocation — per-account tracking with wildcard (*) revocation
  • Config file parsing — custom lexer/parser ported from Go; supports includes, variables, nested blocks, size suffixes
  • Hot reload (SIGHUP) — re-parses config, diffs changes, validates reloadable set, applies with CLI precedence
  • SYSTEM client type — InternalClient with InternalEventSystem, Channel-based send/receive loops, event publishing
  • ACCOUNT client type — lazy per-account InternalClient with import/export subscription support
  • System event publishing — connect/disconnect advisories, server stats, shutdown/lame-duck events, auth errors
  • System request-reply services — $SYS.REQ.SERVER.*.VARZ/CONNZ/SUBSZ/HEALTHZ/IDZ/STATSZ with ping wildcards
  • Account exports/imports — service and stream imports with ExportAuth, subject transforms, response routing, latency tracking

Remaining Lower Priority

  1. Dynamic buffer sizing — delegated to Pipe, less optimized for long-lived connections