From 2baf8a85bfd2bd7f5d6b531e817f027ee1970358 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Sun, 22 Feb 2026 23:59:19 -0500 Subject: [PATCH] docs: update differences.md section 2 to reflect implemented features --- differences.md | 402 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 402 insertions(+) create mode 100644 differences.md diff --git a/differences.md b/differences.md new file mode 100644 index 0000000..2e10971 --- /dev/null +++ b/differences.md @@ -0,0 +1,402 @@ +# Go vs .NET NATS Server: Functionality Differences + +> Excludes clustering/routes, gateways, leaf nodes, and JetStream. +> Generated 2026-02-22 by comparing `golang/nats-server/server/` against `src/NATS.Server/`. + +--- + +## 1. Core Server Lifecycle + +### Server Initialization +| Feature | Go | .NET | Notes | +|---------|:--:|:----:|-------| +| NKey generation (server identity) | Y | Y | Ed25519 key pair via NATS.NKeys at startup | +| System account setup | Y | Y | `$SYS` account created; no event publishing yet (stub) | +| Config file validation on startup | Y | Stub | `-c` flag parsed, `ConfigFile` stored, but no config parser | +| PID file writing | Y | Y | Written on startup, deleted on shutdown | +| Profiling HTTP endpoint (`/debug/pprof`) | Y | Stub | `ProfPort` option exists but endpoint not implemented | +| Ports file output | Y | Y | JSON ports file written to `PortsFileDir` on startup | + +### Accept Loop +| Feature | Go | .NET | Notes | +|---------|:--:|:----:|-------| +| Exponential backoff on accept errors | Y | Y | .NET backs off from 10ms to 1s on repeated failures | +| Config reload lock during client creation | Y | N | Go holds `reloadMu` around `createClient` | +| Goroutine/task tracking (WaitGroup) | Y | Y | `Interlocked` counter + drain with 10s timeout on shutdown | +| Callback-based error handling | Y | N | Go uses `errFunc` callback pattern | +| Random/ephemeral port (port=0) | Y | Y | Port resolved after `Bind`+`Listen`, stored in `_options.Port` | + +### Shutdown +| Feature | Go | .NET | Notes | +|---------|:--:|:----:|-------| +| Graceful shutdown with `WaitForShutdown()` | Y | Y | Idempotent CAS-guarded `ShutdownAsync()` + blocking `WaitForShutdown()` | +| Close reason tracking per connection | Y | Y | 37-value `ClosedState` enum, CAS-based first-writer-wins `MarkClosed()` | +| Lame duck mode (stop new, drain existing) | Y | Y | `LameDuckShutdownAsync()` with grace period + stagger-close with jitter | +| Wait for accept loop completion | Y | Y | `TaskCompletionSource` signaled in accept loop `finally` | +| Flush pending data before close | Y | Y | `FlushAndCloseAsync()` with best-effort flush, skip-flush for error conditions | + +### Signal Handling +| Signal | Go | .NET | Notes | +|--------|:--:|:----:|-------| +| SIGINT (Ctrl+C) | Y | Y | Both handle graceful shutdown | +| SIGTERM | Y | Y | `PosixSignalRegistration` triggers `ShutdownAsync()` | +| SIGUSR1 (reopen logs) | Y | Stub | Signal registered, handler logs "not yet implemented" | +| SIGUSR2 (lame duck mode) | Y | Y | Triggers `LameDuckShutdownAsync()` | +| SIGHUP (config reload) | Y | Stub | Signal registered, handler logs "not yet implemented" | +| Windows Service integration | Y | N | | + +--- + +## 2. Client / Connection Handling + +### Concurrency Model +| Feature | Go | .NET | Notes | +|---------|:--:|:----:|-------| +| Separate read + write loops | Y | Y | Channel-based `RunWriteLoopAsync` with `QueueOutbound()` | +| Write coalescing / batch flush | Y | Y | Write loop drains all channel items before single `FlushAsync` | +| Dynamic buffer sizing (512B-64KB) | Y | N | .NET delegates to `System.IO.Pipelines` | +| Output buffer pooling (3-tier) | Y | N | Go pools at 512B, 4KB, 64KB | + +### Connection Types +| Type | Go | .NET | Notes | +|------|:--:|:----:|-------| +| CLIENT | Y | Y | | +| ROUTER | Y | N | Excluded per scope | +| GATEWAY | Y | N | Excluded per scope | +| LEAF | Y | N | Excluded per scope | +| SYSTEM (internal) | Y | N | | +| JETSTREAM (internal) | Y | N | | +| ACCOUNT (internal) | Y | N | | +| WebSocket clients | Y | N | | +| MQTT clients | Y | N | | + +### Client Features +| Feature | Go | .NET | Notes | +|---------|:--:|:----:|-------| +| Echo suppression (`echo: false`) | Y | Y | .NET checks echo in delivery path (NatsServer.cs:234,253) | +| Verbose mode (`+OK` responses) | Y | Y | Sends `+OK` after CONNECT, SUB, UNSUB, PUB when `verbose:true` | +| No-responders validation | Y | Y | CONNECT rejects `no_responders` without `headers`; 503 HMSG on no match | +| Slow consumer detection | Y | Y | Pending bytes threshold (64MB) + write deadline timeout (10s) | +| Write deadline / timeout policies | Y | Y | `WriteDeadline` option with `CancellationTokenSource.CancelAfter` on flush | +| RTT measurement | Y | N | Go tracks round-trip time per client | +| Per-client trace mode | Y | N | | +| Detailed close reason tracking | Y | Y | 17-value `ClientClosedReason` enum (single-server subset of Go's 37) | +| Connection state flags (16 flags) | Y | Y | 7-flag `ClientFlagHolder` with `Interlocked.Or`/`And` | + +### Slow Consumer Handling +Go implements a sophisticated slow consumer detection system: +- Tracks `pendingBytes` per client output buffer +- If pending exceeds `maxPending`, enters stall mode (2-5ms waits) +- Total stall capped at 10ms per read cycle +- Closes with `SlowConsumerPendingBytes` or `SlowConsumerWriteDeadline` +- Sets `isSlowConsumer` flag for monitoring + +.NET now implements pending bytes tracking and write deadline enforcement via `Channel>`. Key differences from Go: no stall/retry mode (immediate close on threshold exceeded), write deadline via `CancellationTokenSource.CancelAfter` instead of `SetWriteDeadline`. `IsSlowConsumer` flag and server-level `SlowConsumerCount` stats are tracked for monitoring. + +### Stats Tracking +| Feature | Go | .NET | Notes | +|---------|:--:|:----:|-------| +| Per-connection atomic stats | Y | Y | .NET uses `Interlocked` for stats access | +| Per-read-cycle stat batching | Y | Y | Local accumulators flushed via `Interlocked.Add` per read cycle | +| Per-account stats | Y | N | | +| Slow consumer counters | Y | Y | `SlowConsumers` and `SlowConsumerClients` incremented on detection | + +--- + +## 3. Protocol Parsing + +### Parser Architecture +| Aspect | Go | .NET | +|--------|-----|------| +| Approach | Byte-by-byte state machine (74 states) | Two-phase: line extraction + command dispatch | +| Case handling | Per-state character checks | Bit-mask lowercase normalization (`\| 0x20`) | +| Buffer strategy | Jump-ahead optimization for payloads | Direct size-based reads via Pipe | +| Split-buffer handling | argBuf accumulation with scratch buffer | State variables (`_awaitingPayload`, etc.) | +| Error model | Inline error sending + error return | Exception-based (`ProtocolViolationException`) | +| CRLF in payload | Included in message buffer | Excluded by design | + +### Protocol Operations +| Operation | Go | .NET | Notes | +|-----------|:--:|:----:|-------| +| PUB | Y | Y | | +| HPUB (headers) | Y | Y | | +| SUB | Y | Y | | +| UNSUB | Y | Y | | +| CONNECT | Y | Y | | +| INFO | Y | Y | | +| PING / PONG | Y | Y | | +| MSG / HMSG | Y | Y | | +| +OK / -ERR | Y | Y | | +| RS+/RS-/RMSG (routes) | Y | N | Excluded per scope | +| A+/A- (accounts) | Y | N | Excluded per scope | +| LS+/LS-/LMSG (leaf) | Y | N | Excluded per scope | + +### Protocol Parsing Gaps +| Feature | Go | .NET | Notes | +|---------|:--:|:----:|-------| +| Multi-client-type command routing | Y | N | Go checks `c.kind` to allow/reject commands | +| Protocol tracing in parser | Y | N | Go calls `traceInOp()` per operation | +| Subject mapping (input→output) | Y | N | Go transforms subjects via mapping rules | +| MIME header parsing | Y | N | .NET delegates header handling to client layer | +| Message trace event initialization | Y | N | | + +### Protocol Writing +| Aspect | Go | .NET | Notes | +|--------|:--:|:----:|-------| +| INFO serialization | Once at startup | Every send | .NET re-serializes JSON each time | +| MSG/HMSG construction | Direct buffer write | String interpolation → byte encode | More allocations in .NET | +| Pre-encoded constants | Y | Y | Both pre-encode PING/PONG/OK | + +--- + +## 4. Subscriptions & Subject Matching + +### Trie Implementation +| Feature | Go | .NET | Notes | +|---------|:--:|:----:|-------| +| Basic trie with `*`/`>` wildcards | Y | Y | Core matching identical | +| Queue group support | Y | Y | | +| Result caching (1024 max) | Y | Y | Same limits | +| `plist` optimization (>256 subs) | Y | N | Go converts high-fanout nodes to array | +| Async cache sweep (background) | Y | N | .NET sweeps inline under write lock | +| Atomic generation ID for invalidation | Y | N | .NET clears cache explicitly | +| Cache eviction strategy | Random | First-N | Semantic difference minimal | + +### Missing SubList Features +| Feature | Go | .NET | Notes | +|---------|:--:|:----:|-------| +| `Stats()` — comprehensive statistics | Y | N | Matches, cache hits, inserts, removes, fanout | +| `HasInterest()` — fast bool check | Y | N | | +| `NumInterest()` — fast count | Y | N | | +| `ReverseMatch()` — pattern→literal query | Y | N | | +| `RemoveBatch()` — efficient bulk removal | Y | N | | +| `All()` — enumerate all subscriptions | Y | N | | +| Notification system (interest changes) | Y | N | | +| Local/remote subscription filtering | Y | N | | +| Queue weight expansion (remote subs) | Y | N | | +| `MatchBytes()` — zero-copy byte API | Y | N | | + +### Subject Validation +| Feature | Go | .NET | Notes | +|---------|:--:|:----:|-------| +| Basic validation (empty tokens, wildcards) | Y | Y | | +| Literal subject check | Y | Y | | +| UTF-8/null rune validation | Y | N | Go has `checkRunes` parameter | +| Collision detection (`SubjectsCollide`) | Y | N | | +| Token utilities (`tokenAt`, `numTokens`) | Y | N | | +| Stack-allocated token buffer | Y | N | Go uses `[32]string{}` on stack | + +### Subscription Lifecycle +| Feature | Go | .NET | Notes | +|---------|:--:|:----:|-------| +| Per-account subscription limit | Y | N | | +| Auto-unsubscribe on max messages | Y | Y | .NET enforces at delivery time (NatsServer.cs:269-270) | +| Subscription routing propagation | Y | N | For clusters | +| Queue weight (`qw`) field | Y | N | For remote queue load balancing | + +--- + +## 5. Authentication & Authorization + +### Auth Mechanisms +| Mechanism | Go | .NET | Notes | +|-----------|:--:|:----:|-------| +| Username/password | Y | Y | | +| Token | Y | Y | | +| NKeys (Ed25519) | Y | Y | .NET has framework but integration is basic | +| JWT validation | Y | N | | +| Bcrypt password hashing | Y | Y | .NET supports bcrypt (`$2*` prefix) with constant-time fallback | +| TLS certificate mapping | Y | N | Property exists but no implementation | +| Custom auth interface | Y | N | | +| External auth callout | Y | N | | +| Proxy authentication | Y | N | | +| Bearer tokens | Y | N | | +| User revocation tracking | Y | N | | + +### Account System +| Feature | Go | .NET | Notes | +|---------|:--:|:----:|-------| +| Per-account SubList isolation | Y | Y | | +| Multi-account user resolution | Y | N | .NET has basic account, no resolution | +| Account exports/imports | Y | N | | +| Per-account connection limits | Y | N | | +| Per-account subscription limits | Y | N | | +| Account JetStream limits | Y | N | Excluded per scope | + +### Permissions +| Feature | Go | .NET | Notes | +|---------|:--:|:----:|-------| +| Publish allow list | Y | Y | | +| Subscribe allow list | Y | Y | | +| Publish deny list | Y | Partial | .NET has deny in struct but limited enforcement | +| Subscribe deny list | Y | Partial | Same | +| Message-level deny filtering | Y | N | Go filters at delivery time | +| Permission caching (128 entries) | Y | N | | +| Response permissions (reply tracking) | Y | N | Dynamic reply subject authorization | +| Permission templates (JWT) | Y | N | e.g., `{{name()}}`, `{{account-tag(...)}}` | + +--- + +## 6. Configuration + +### CLI Flags +| Flag | Go | .NET | Notes | +|------|:--:|:----:|-------| +| `-p/--port` | Y | Y | | +| `-a/--addr` | Y | Y | | +| `-n/--name` (ServerName) | Y | Y | | +| `-m/--http_port` (monitoring) | Y | Y | | +| `-c` (config file) | Y | Stub | Flag parsed, stored in `ConfigFile`, no config parser | +| `-D/-V/-DV` (debug/trace) | Y | N | | +| `--tlscert/--tlskey/--tlscacert` | Y | Y | | +| `--tlsverify` | Y | Y | | +| `--http_base_path` | Y | Y | | +| `--https_port` | Y | Y | | + +### Configuration System +| Feature | Go | .NET | Notes | +|---------|:--:|:----:|-------| +| Config file parsing | Y | N | Go has custom `conf` parser with includes | +| Hot reload (SIGHUP) | Y | N | | +| Config change detection | Y | N | Go tracks `inConfig`/`inCmdLine` origins | +| ~450 option fields | Y | ~54 | .NET covers core options only | + +### Missing Options Categories +- Logging options (file, rotation, syslog, trace levels) +- Advanced limits (MaxSubs, MaxSubTokens, MaxPending, WriteDeadline) +- Tags/metadata +- OCSP configuration +- WebSocket/MQTT options +- Operator mode / account resolver + +--- + +## 7. Monitoring + +### HTTP Endpoints +| Endpoint | Go | .NET | Notes | +|----------|:--:|:----:|-------| +| `/healthz` | Y | Y | | +| `/varz` | Y | Y | | +| `/connz` | Y | Y | | +| `/` (root listing) | Y | Y | | +| `/routez` | Y | Stub | Returns empty response | +| `/gatewayz` | Y | Stub | Returns empty response | +| `/leafz` | Y | Stub | Returns empty response | +| `/subz` / `/subscriptionsz` | Y | Stub | Returns empty response | +| `/accountz` | Y | Stub | Returns empty response | +| `/accstatz` | Y | Stub | Returns empty response | +| `/jsz` | Y | Stub | Returns empty response | + +### Varz Response +| Field Category | Go | .NET | Notes | +|----------------|:--:|:----:|-------| +| Identity (ID, Name, Version) | Y | Y | | +| Network (Host, Port, URLs) | Y | Y | | +| Security (AuthRequired, TLS) | Y | Y | | +| Limits (MaxConn, MaxPayload) | Y | Y | | +| Timing (Start, Now, Uptime) | Y | Y | | +| Runtime (Mem, CPU, Cores) | Y | Y | | +| Connections (current, total) | Y | Y | | +| Messages (in/out msgs/bytes) | Y | Y | | +| SlowConsumer breakdown | Y | N | Go tracks per connection type | +| Cluster/Gateway/Leaf blocks | Y | N | Excluded per scope | +| JetStream block | Y | N | Excluded per scope | +| TLS cert expiry info | Y | N | | + +### Connz Response +| Feature | Go | .NET | Notes | +|---------|:--:|:----:|-------| +| Filtering by CID, user, account | Y | Partial | | +| Sorting (11 options) | Y | Y | .NET missing ByStop, ByReason | +| Pagination (offset, limit) | Y | Y | | +| Subscription detail mode | Y | N | | +| TLS peer certificate info | Y | N | | +| JWT/IssuerKey/Tags fields | Y | N | | +| MQTT client ID filtering | Y | N | | +| Proxy info | Y | N | | + +--- + +## 8. TLS + +### TLS Modes +| Mode | Go | .NET | Notes | +|------|:--:|:----:|-------| +| No TLS | Y | Y | | +| INFO-first (default NATS) | Y | Y | | +| TLS-first (before INFO) | Y | Y | | +| Mixed/Fallback | Y | Y | | +| TLS-required | Y | Y | | + +### TLS Features +| Feature | Go | .NET | Notes | +|---------|:--:|:----:|-------| +| PEM cert/key loading | Y | Y | | +| CA chain validation | Y | Y | | +| Mutual TLS (client certs) | Y | Y | | +| Certificate pinning (SHA256 SPKI) | Y | Y | | +| TLS handshake timeout | Y | Y | | +| TLS rate limiting | Y | Property only | .NET has the option but enforcement is partial | +| First-byte peeking (0x16 detection) | Y | Y | | +| Cert subject→user mapping | Y | N | `TlsMap` property exists, no implementation | +| OCSP stapling | Y | N | | +| Min TLS version control | Y | Y | | + +--- + +## 9. Logging + +| Feature | Go | .NET | Notes | +|---------|:--:|:----:|-------| +| Structured logging | Partial | Y | .NET uses Serilog with ILogger | +| File logging with rotation | Y | N | | +| Syslog (local and remote) | Y | N | | +| Log reopening (SIGUSR1) | Y | N | | +| Trace mode (protocol-level) | Y | N | | +| Debug mode | Y | N | | +| Per-subsystem log control | Y | N | | +| Color output on TTY | Y | N | | +| Timestamp format control | Y | N | | + +--- + +## 10. Ping/Pong & Keepalive + +| Feature | Go | .NET | Notes | +|---------|:--:|:----:|-------| +| Server-initiated PING | Y | Y | | +| Configurable interval | Y | Y | PingInterval option | +| Max pings out | Y | Y | MaxPingsOut option | +| Stale connection close | Y | Y | | +| RTT-based first PING delay | Y | N | Go delays first PING based on RTT | +| RTT tracking | Y | N | | +| Stale connection watcher | Y | N | Go has dedicated watcher goroutine | + +--- + +## Summary: Critical Gaps for Production Use + +### High Priority +1. **Slow consumer detection** — unbounded writes can exhaust memory (stat fields exist but no detection logic) +2. **Write coalescing / batch flush** — performance gap for high-throughput scenarios + +### Medium Priority +3. **Verbose mode** — clients expect `+OK` when `verbose: true` +4. **Permission deny enforcement at delivery** — deny lists checked at SUB/PUB time but not during message delivery +5. **Config file parsing** — needed for production deployment (CLI stub exists) +6. **Hot reload** — needed for zero-downtime config changes (SIGHUP stub exists) +7. **File logging with rotation** — needed for production logging +8. **No-responders validation** — flag parsed but not enforced + +### Lower Priority +9. **Dynamic buffer sizing** — delegated to Pipe, less optimized for long-lived connections +10. **JWT authentication** — needed for operator mode +11. **TLS certificate mapping** — property exists, not implemented +12. **OCSP support** — certificate revocation checking +13. **Subject mapping** — input→output subject transformation +14. **Protocol tracing** — no trace-level logging +15. **Subscription statistics** — SubList has no stats collection +16. **Per-account limits** — connections, subscriptions per account +17. **Reply subject tracking** — dynamic response permissions +18. **Windows Service integration** — needed for Windows deployment