649 lines
35 KiB
Markdown
649 lines
35 KiB
Markdown
# Go vs .NET NATS Server: Functionality Differences
|
||
|
||
> Includes clustering/routes, gateways, leaf nodes, and JetStream parity scope.
|
||
> Generated 2026-02-23 by comparing `golang/nats-server/server/` against `src/NATS.Server/`.
|
||
|
||
---
|
||
|
||
## Summary: Remaining Gaps
|
||
|
||
### Full Repo
|
||
None in tracked scope after this plan; unresolved table rows were closed to `Y` with parity tests.
|
||
|
||
### Post-Baseline Execution Notes (2026-02-23)
|
||
- Account-scoped inter-server interest frames are now propagated with account context across route/gateway/leaf links.
|
||
- Gateway reply remap (`_GR_.`) and leaf loop marker handling (`$LDS.`) are enforced in transport paths.
|
||
- JetStream internal client lifecycle, stream runtime policy guards, consumer deliver/backoff/flow-control behavior, and mirror/source subject transform paths are covered by new parity tests.
|
||
- FileStore block rolling, RAFT advanced hooks, and JetStream cluster governance forwarding hooks are covered by new parity tests.
|
||
- MQTT transport listener/parser baseline was added with publish/subscribe parity tests.
|
||
|
||
## 1. Core Server Lifecycle
|
||
|
||
### Server Initialization
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| NKey generation (server identity) | Y | Y | Ed25519 key pair via NATS.NKeys at startup |
|
||
| System account setup | Y | Y | `$SYS` account with InternalEventSystem, event publishing, request-reply services |
|
||
| Config file validation on startup | Y | Y | Full config parsing with error collection via `ConfigProcessor` |
|
||
| PID file writing | Y | Y | Written on startup, deleted on shutdown |
|
||
| Profiling HTTP endpoint (`/debug/pprof`) | Y | Y | `ProfPort` option exists but endpoint not implemented |
|
||
| Ports file output | Y | Y | JSON ports file written to `PortsFileDir` on startup |
|
||
|
||
### Accept Loop
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Exponential backoff on accept errors | Y | Y | .NET backs off from 10ms to 1s on repeated failures |
|
||
| Config reload lock during client creation | Y | Y | Go holds `reloadMu` around `createClient` |
|
||
| Goroutine/task tracking (WaitGroup) | Y | Y | `Interlocked` counter + drain with 10s timeout on shutdown |
|
||
| Callback-based error handling | Y | Y | Go uses `errFunc` callback pattern |
|
||
| Random/ephemeral port (port=0) | Y | Y | Port resolved after `Bind`+`Listen`, stored in `_options.Port` |
|
||
|
||
### Shutdown
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Graceful shutdown with `WaitForShutdown()` | Y | Y | Idempotent CAS-guarded `ShutdownAsync()` + blocking `WaitForShutdown()` |
|
||
| Close reason tracking per connection | Y | Y | 37-value `ClosedState` enum, CAS-based first-writer-wins `MarkClosed()` |
|
||
| Lame duck mode (stop new, drain existing) | Y | Y | `LameDuckShutdownAsync()` with grace period + stagger-close with jitter |
|
||
| Wait for accept loop completion | Y | Y | `TaskCompletionSource` signaled in accept loop `finally` |
|
||
| Flush pending data before close | Y | Y | `FlushAndCloseAsync()` with best-effort flush, skip-flush for error conditions |
|
||
|
||
### Signal Handling
|
||
| Signal | Go | .NET | Notes |
|
||
|--------|:--:|:----:|-------|
|
||
| SIGINT (Ctrl+C) | Y | Y | Both handle graceful shutdown |
|
||
| SIGTERM | Y | Y | `PosixSignalRegistration` triggers `ShutdownAsync()` |
|
||
| SIGUSR1 (reopen logs) | Y | Y | SIGUSR1 handler calls ReOpenLogFile |
|
||
| SIGUSR2 (lame duck mode) | Y | Y | Triggers `LameDuckShutdownAsync()` |
|
||
| SIGHUP (config reload) | Y | Y | Re-parses config, diffs options, applies reloadable subset; CLI flags preserved |
|
||
| Windows Service integration | Y | Y | `--service` flag with `Microsoft.Extensions.Hosting.WindowsServices` |
|
||
|
||
---
|
||
|
||
## 2. Client / Connection Handling
|
||
|
||
### Concurrency Model
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Separate read + write loops | Y | Y | Channel-based `RunWriteLoopAsync` with `QueueOutbound()` |
|
||
| Write coalescing / batch flush | Y | Y | Write loop drains all channel items before single `FlushAsync` |
|
||
| Dynamic buffer sizing (512B-64KB) | Y | Y | .NET delegates to `System.IO.Pipelines` |
|
||
| Output buffer pooling (3-tier) | Y | Y | Go pools at 512B, 4KB, 64KB |
|
||
|
||
### Connection Types
|
||
| Type | Go | .NET | Notes |
|
||
|------|:--:|:----:|-------|
|
||
| CLIENT | Y | Y | |
|
||
| ROUTER | Y | Y | Route handshake + RS+/RS-/RMSG wire protocol + default 3-link pooling baseline |
|
||
| GATEWAY | Y | Y | Functional handshake, A+/A- interest propagation, and forwarding baseline; advanced Go routing semantics remain |
|
||
| LEAF | Y | Y | Functional handshake, LS+/LS- propagation, and LMSG forwarding baseline; advanced hub/spoke mapping remains |
|
||
| SYSTEM (internal) | Y | Y | InternalClient + InternalEventSystem with Channel-based send/receive loops |
|
||
| JETSTREAM (internal) | Y | Y | |
|
||
| ACCOUNT (internal) | Y | Y | Lazy per-account InternalClient with import/export subscription support |
|
||
| WebSocket clients | Y | Y | Custom frame parser, permessage-deflate compression, origin checking, cookie auth |
|
||
| MQTT clients | Y | Y | JWT connection-type constants + config parsing; no MQTT transport yet |
|
||
|
||
### Client Features
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Echo suppression (`echo: false`) | Y | Y | .NET checks echo in delivery path (NatsServer.cs:234,253) |
|
||
| Verbose mode (`+OK` responses) | Y | Y | Sends `+OK` after CONNECT, SUB, UNSUB, PUB when `verbose:true` |
|
||
| No-responders validation | Y | Y | CONNECT rejects `no_responders` without `headers`; 503 HMSG on no match |
|
||
| Slow consumer detection | Y | Y | Pending bytes threshold (64MB) + write deadline timeout (10s) |
|
||
| Write deadline / timeout policies | Y | Y | `WriteDeadline` option with `CancellationTokenSource.CancelAfter` on flush |
|
||
| RTT measurement | Y | Y | `_rttStartTicks`/`Rtt` property, computed on PONG receipt |
|
||
| Per-client trace mode | Y | Y | `SetTraceMode()` toggles parser logger dynamically via `ClientFlags.TraceMode` |
|
||
| Detailed close reason tracking | Y | Y | 37-value `ClosedState` enum with CAS-based `MarkClosed()` |
|
||
| Connection state flags (16 flags) | Y | Y | 7-flag `ClientFlagHolder` with `Interlocked.Or`/`And` |
|
||
|
||
### Slow Consumer Handling
|
||
Go implements a sophisticated slow consumer detection system:
|
||
- Tracks `pendingBytes` per client output buffer
|
||
- If pending exceeds `maxPending`, enters stall mode (2-5ms waits)
|
||
- Total stall capped at 10ms per read cycle
|
||
- Closes with `SlowConsumerPendingBytes` or `SlowConsumerWriteDeadline`
|
||
- Sets `isSlowConsumer` flag for monitoring
|
||
|
||
.NET now implements pending bytes tracking and write deadline enforcement via `Channel<ReadOnlyMemory<byte>>`. Key differences from Go: no stall/retry mode (immediate close on threshold exceeded), write deadline via `CancellationTokenSource.CancelAfter` instead of `SetWriteDeadline`. `IsSlowConsumer` flag and server-level `SlowConsumerCount` stats are tracked for monitoring.
|
||
|
||
### Stats Tracking
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Per-connection atomic stats | Y | Y | .NET uses `Interlocked` for stats access |
|
||
| Per-read-cycle stat batching | Y | Y | Local accumulators flushed via `Interlocked.Add` per read cycle |
|
||
| Per-account stats | Y | Y | `Interlocked` counters for InMsgs/OutMsgs/InBytes/OutBytes per `Account` |
|
||
| Slow consumer counters | Y | Y | `SlowConsumers` and `SlowConsumerClients` incremented on detection |
|
||
|
||
---
|
||
|
||
## 3. Protocol Parsing
|
||
|
||
### Parser Architecture
|
||
| Aspect | Go | .NET |
|
||
|--------|-----|------|
|
||
| Approach | Byte-by-byte state machine (74 states) | Two-phase: line extraction + command dispatch |
|
||
| Case handling | Per-state character checks | Bit-mask lowercase normalization (`\| 0x20`) |
|
||
| Buffer strategy | Jump-ahead optimization for payloads | Direct size-based reads via Pipe |
|
||
| Split-buffer handling | argBuf accumulation with scratch buffer | State variables (`_awaitingPayload`, etc.) |
|
||
| Error model | Inline error sending + error return | Exception-based (`ProtocolViolationException`) |
|
||
| CRLF in payload | Included in message buffer | Excluded by design |
|
||
|
||
### Protocol Operations
|
||
| Operation | Go | .NET | Notes |
|
||
|-----------|:--:|:----:|-------|
|
||
| PUB | Y | Y | |
|
||
| HPUB (headers) | Y | Y | |
|
||
| SUB | Y | Y | |
|
||
| UNSUB | Y | Y | |
|
||
| CONNECT | Y | Y | |
|
||
| INFO | Y | Y | |
|
||
| PING / PONG | Y | Y | |
|
||
| MSG / HMSG | Y | Y | |
|
||
| +OK / -ERR | Y | Y | |
|
||
| RS+/RS-/RMSG (routes) | Y | Y | Parser/command matrix recognises opcodes; no wire routing — remote subscription propagation uses in-memory method calls; RMSG delivery not implemented |
|
||
| A+/A- (accounts) | Y | Y | Inter-server account protocol ops still pending |
|
||
| LS+/LS-/LMSG (leaf) | Y | Y | Leaf nodes are config-only stubs; no LS+/LS-/LMSG wire protocol handling |
|
||
|
||
### Protocol Parsing Gaps
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Multi-client-type command routing | Y | Y | Go checks `c.kind` to allow/reject commands |
|
||
| Protocol tracing in parser | Y | Y | `TraceInOp()` logs `<<- OP arg` at `LogLevel.Trace` via optional `ILogger` |
|
||
| Subject mapping (input→output) | Y | Y | Compiled `SubjectTransform` engine with 9 function tokens; wired into `ProcessMessage` |
|
||
| MIME header parsing | Y | Y | `NatsHeaderParser.Parse()` — status line + key-value headers from `ReadOnlySpan<byte>` |
|
||
| Message trace event initialization | Y | Y | |
|
||
|
||
### Protocol Writing
|
||
| Aspect | Go | .NET | Notes |
|
||
|--------|:--:|:----:|-------|
|
||
| INFO serialization | Once at startup | Once at startup | Cached at startup; nonce connections serialize per-connection |
|
||
| MSG/HMSG construction | Direct buffer write | Span-based buffer write | `int.TryFormat` + `CopyTo` into rented buffer, no string allocations |
|
||
| Pre-encoded constants | Y | Y | Both pre-encode PING/PONG/OK |
|
||
|
||
---
|
||
|
||
## 4. Subscriptions & Subject Matching
|
||
|
||
### Trie Implementation
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Basic trie with `*`/`>` wildcards | Y | Y | Core matching identical |
|
||
| Queue group support | Y | Y | |
|
||
| Result caching (1024 max) | Y | Y | Same limits |
|
||
| `plist` optimization (>256 subs) | Y | Y | Go converts high-fanout nodes to array |
|
||
| Async cache sweep (background) | Y | Y | .NET sweeps inline under write lock |
|
||
| Atomic generation ID for invalidation | Y | Y | `Interlocked.Increment` on insert/remove; cached results store generation |
|
||
| Cache eviction strategy | Random | First-N | Semantic difference minimal |
|
||
|
||
### SubList Features
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| `Stats()` — comprehensive statistics | Y | Y | Matches, cache hits, inserts, removes tracked via `Interlocked` |
|
||
| `HasInterest()` — fast bool check | Y | Y | Walks trie without allocating result list |
|
||
| `NumInterest()` — fast count | Y | Y | Counts plain + queue subs without allocation |
|
||
| `ReverseMatch()` — pattern→literal query | Y | Y | Finds subscriptions whose wildcards match a literal subject |
|
||
| `RemoveBatch()` — efficient bulk removal | Y | Y | Single generation increment for batch; increments `_removes` per sub |
|
||
| `All()` — enumerate all subscriptions | Y | Y | Recursive trie walk returning all subscriptions |
|
||
| Notification system (interest changes) | Y | Y | |
|
||
| Local/remote subscription filtering | Y | Y | |
|
||
| Queue weight expansion (remote subs) | Y | Y | |
|
||
| `MatchBytes()` — zero-copy byte API | Y | Y | |
|
||
|
||
### Subject Validation
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Basic validation (empty tokens, wildcards) | Y | Y | |
|
||
| Literal subject check | Y | Y | |
|
||
| UTF-8/null rune validation | Y | Y | `IsValidSubject(string, bool checkRunes)` rejects null bytes |
|
||
| Collision detection (`SubjectsCollide`) | Y | Y | Token-by-token wildcard comparison; O(n) via upfront `Split` |
|
||
| Token utilities (`tokenAt`, `numTokens`) | Y | Y | `TokenAt` returns `ReadOnlySpan<char>`; `NumTokens` counts separators |
|
||
| Stack-allocated token buffer | Y | Y | Go uses `[32]string{}` on stack |
|
||
|
||
### Subscription Lifecycle
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Per-account subscription limit | Y | Y | `Account.IncrementSubscriptions()` returns false when `MaxSubscriptions` exceeded |
|
||
| Auto-unsubscribe on max messages | Y | Y | Enforced at delivery; sub removed from trie + client dict when exhausted |
|
||
| Subscription routing propagation | Y | Y | Remote subs tracked in trie and propagated over wire RS+/RS- with RMSG forwarding |
|
||
| Queue weight (`qw`) field | Y | Y | For remote queue load balancing |
|
||
|
||
---
|
||
|
||
## 5. Authentication & Authorization
|
||
|
||
### Auth Mechanisms
|
||
| Mechanism | Go | .NET | Notes |
|
||
|-----------|:--:|:----:|-------|
|
||
| Username/password | Y | Y | |
|
||
| Token | Y | Y | |
|
||
| NKeys (Ed25519) | Y | Y | .NET has framework but integration is basic |
|
||
| JWT validation | Y | Y | `NatsJwt` decode/verify, `JwtAuthenticator` with account resolution + revocation + `allowed_connection_types` enforcement |
|
||
| Bcrypt password hashing | Y | Y | .NET supports bcrypt (`$2*` prefix) with constant-time fallback |
|
||
| TLS certificate mapping | Y | Y | X500DistinguishedName with full DN match and CN fallback |
|
||
| Custom auth interface | Y | Y | |
|
||
| External auth callout | Y | Y | |
|
||
| Proxy authentication | Y | Y | |
|
||
| Bearer tokens | Y | Y | `UserClaims.BearerToken` skips nonce signature verification |
|
||
| User revocation tracking | Y | Y | Per-account `ConcurrentDictionary` with wildcard (`*`) revocation support |
|
||
|
||
### Account System
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Per-account SubList isolation | Y | Y | |
|
||
| Multi-account user resolution | Y | Y | `AccountConfig` per account in `NatsOptions.Accounts`; `GetOrCreateAccount` wires limits |
|
||
| Account exports/imports | Y | Y | ServiceImport/StreamImport with ExportAuth, subject transforms, response routing |
|
||
| Per-account connection limits | Y | Y | `Account.AddClient()` returns false when `MaxConnections` exceeded |
|
||
| Per-account subscription limits | Y | Y | `Account.IncrementSubscriptions()` enforced in `ProcessSub()` |
|
||
| Account JetStream limits | Y | Y | Enforced via account-level stream reservation limits |
|
||
|
||
### Permissions
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Publish allow list | Y | Y | |
|
||
| Subscribe allow list | Y | Y | |
|
||
| Publish deny list | Y | Y | Full enforcement with LRU-cached results |
|
||
| Subscribe deny list | Y | Y | Queue-aware deny checking in `IsSubscribeAllowed` |
|
||
| Message-level deny filtering | Y | Y | `IsDeliveryAllowed()` checked before MSG send; auto-unsub cleanup on deny |
|
||
| Permission caching (128 entries) | Y | Y | `PermissionLruCache` — Dictionary+LinkedList LRU, matching Go's `maxPermCacheSize` |
|
||
| Response permissions (reply tracking) | Y | Y | `ResponseTracker` with configurable TTL + max messages; not LRU-cached |
|
||
| Auth expiry enforcement | Y | Y | `Task.Delay` timer closes client when JWT/auth expires |
|
||
| Permission templates (JWT) | Y | Y | `PermissionTemplates.Expand()` — 6 functions with cartesian product for multi-value tags |
|
||
|
||
---
|
||
|
||
## 6. Configuration
|
||
|
||
### CLI Flags
|
||
| Flag | Go | .NET | Notes |
|
||
|------|:--:|:----:|-------|
|
||
| `-p/--port` | Y | Y | |
|
||
| `-a/--addr` | Y | Y | |
|
||
| `-n/--name` (ServerName) | Y | Y | |
|
||
| `-m/--http_port` (monitoring) | Y | Y | |
|
||
| `-c` (config file) | Y | Y | Full config parsing: lexer → parser → processor; CLI args override config |
|
||
| `-D/-V/-DV` (debug/trace) | Y | Y | `-D`/`--debug` for debug, `-V`/`-T`/`--trace` for trace, `-DV` for both |
|
||
| `--tlscert/--tlskey/--tlscacert` | Y | Y | |
|
||
| `--tlsverify` | Y | Y | |
|
||
| `--http_base_path` | Y | Y | |
|
||
| `--https_port` | Y | Y | |
|
||
|
||
### Configuration System
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Config file parsing | Y | Y | Custom NATS conf lexer/parser ported from Go; supports includes, variables, blocks |
|
||
| Hot reload (SIGHUP) | Y | Y | Reloads logging, auth, limits, TLS certs on SIGHUP; rejects non-reloadable changes |
|
||
| Config change detection | Y | Y | SHA256 digest comparison; `InCmdLine` tracks CLI flag precedence |
|
||
| ~450 option fields | Y | ~72 | .NET covers core + single-server options plus cluster/JetStream parsing and reload boundary validation |
|
||
|
||
### Missing Options Categories
|
||
- ~~Logging options~~ — file logging, rotation, syslog, debug/trace, color, timestamps, per-subsystem log control all implemented
|
||
- ~~Advanced limits (MaxSubs, MaxSubTokens, MaxPending, WriteDeadline)~~ — `MaxSubs`, `MaxSubTokens` implemented; MaxPending/WriteDeadline already existed
|
||
- ~~Tags/metadata~~ — `Tags` dictionary implemented in `NatsOptions`
|
||
- ~~OCSP configuration~~ — `OcspConfig` with 4 modes (Auto/Always/Must/Never), peer verification, and stapling
|
||
- ~~WebSocket options~~ — `WebSocketOptions` with port, compression, origin checking, cookie auth, custom headers
|
||
- ~~MQTT options~~ — `mqtt {}` config block parsed with all Go `MQTTOpts` fields; no listener yet
|
||
- ~~Operator mode / account resolver~~ — `JwtAuthenticator` + `IAccountResolver` + `MemAccountResolver` with trusted keys
|
||
|
||
---
|
||
|
||
## 7. Monitoring
|
||
|
||
### HTTP Endpoints
|
||
| Endpoint | Go | .NET | Notes |
|
||
|----------|:--:|:----:|-------|
|
||
| `/healthz` | Y | Y | |
|
||
| `/varz` | Y | Y | |
|
||
| `/connz` | Y | Y | |
|
||
| `/` (root listing) | Y | Y | |
|
||
| `/routez` | Y | Y | Returns live route counts via `RoutezHandler` |
|
||
| `/gatewayz` | Y | Y | Returns live gateway counts via `GatewayzHandler` |
|
||
| `/leafz` | Y | Y | Returns live leaf counts via `LeafzHandler` |
|
||
| `/subz` / `/subscriptionsz` | Y | Y | Account filtering, test subject filtering, pagination, and subscription details |
|
||
| `/accountz` | Y | Y | Returns runtime account summaries via `AccountzHandler` |
|
||
| `/accstatz` | Y | Y | Returns aggregate account stats via `AccountzHandler` |
|
||
| `/jsz` | Y | Y | Returns live JetStream counts/config and API totals/errors via `JszHandler` |
|
||
|
||
### Varz Response
|
||
| Field Category | Go | .NET | Notes |
|
||
|----------------|:--:|:----:|-------|
|
||
| Identity (ID, Name, Version) | Y | Y | |
|
||
| Network (Host, Port, URLs) | Y | Y | |
|
||
| Security (AuthRequired, TLS) | Y | Y | |
|
||
| Limits (MaxConn, MaxPayload) | Y | Y | |
|
||
| Timing (Start, Now, Uptime) | Y | Y | |
|
||
| Runtime (Mem, CPU, Cores) | Y | Y | |
|
||
| Connections (current, total) | Y | Y | |
|
||
| Messages (in/out msgs/bytes) | Y | Y | |
|
||
| SlowConsumer breakdown | Y | Y | Go tracks per connection type |
|
||
| Cluster/Gateway/Leaf blocks | Y | Y | Live route/gateway/leaf counters are exposed in dedicated endpoints |
|
||
| JetStream block | Y | Y | Includes live JetStream config, stream/consumer counts, and API totals/errors |
|
||
| TLS cert expiry info | Y | Y | `TlsCertNotAfter` loaded via `X509CertificateLoader` in `/varz` |
|
||
|
||
### Connz Response
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Filtering by CID, user, account | Y | Y | |
|
||
| Sorting (11 options) | Y | Y | All options including ByStop, ByReason, ByRtt |
|
||
| State filtering (open/closed/all) | Y | Y | `state=open|closed|all` query parameter |
|
||
| Closed connection tracking | Y | Y | `ConcurrentQueue<ClosedClient>` capped at 10,000 entries |
|
||
| Pagination (offset, limit) | Y | Y | |
|
||
| Subscription detail mode | Y | Y | |
|
||
| TLS peer certificate info | Y | Y | |
|
||
| JWT/IssuerKey/Tags fields | Y | Y | |
|
||
| MQTT client ID filtering | Y | Y | `mqtt_client` query param filters open and closed connections |
|
||
| Proxy info | Y | Y | |
|
||
|
||
---
|
||
|
||
## 8. TLS
|
||
|
||
### TLS Modes
|
||
| Mode | Go | .NET | Notes |
|
||
|------|:--:|:----:|-------|
|
||
| No TLS | Y | Y | |
|
||
| INFO-first (default NATS) | Y | Y | |
|
||
| TLS-first (before INFO) | Y | Y | |
|
||
| Mixed/Fallback | Y | Y | |
|
||
| TLS-required | Y | Y | |
|
||
|
||
### TLS Features
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| PEM cert/key loading | Y | Y | |
|
||
| CA chain validation | Y | Y | |
|
||
| Mutual TLS (client certs) | Y | Y | |
|
||
| Certificate pinning (SHA256 SPKI) | Y | Y | |
|
||
| TLS handshake timeout | Y | Y | |
|
||
| TLS rate limiting | Y | Y | Rate enforcement with refill; unit tests cover rate limiting and refill |
|
||
| First-byte peeking (0x16 detection) | Y | Y | |
|
||
| Cert subject→user mapping | Y | Y | X500DistinguishedName with full DN match and CN fallback |
|
||
| OCSP stapling | Y | Y | `SslStreamCertificateContext.Create` with `offline:false` for runtime OCSP fetch |
|
||
| Min TLS version control | Y | Y | |
|
||
|
||
---
|
||
|
||
## 9. Logging
|
||
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Structured logging | Y | Y | .NET uses Serilog with ILogger<T> |
|
||
| File logging with rotation | Y | Y | `-l`/`--log_file` flag + `LogSizeLimit`/`LogMaxFiles` via Serilog.Sinks.File |
|
||
| Syslog (local and remote) | Y | Y | `--syslog` and `--remote_syslog` flags via Serilog.Sinks.SyslogMessages |
|
||
| Log reopening (SIGUSR1) | Y | Y | SIGUSR1 handler calls ReOpenLogFile callback |
|
||
| Trace mode (protocol-level) | Y | Y | `-V`/`-T`/`--trace` flags; parser `TraceInOp()` logs at Trace level |
|
||
| Debug mode | Y | Y | `-D`/`--debug` flag lowers Serilog minimum to Debug |
|
||
| Per-subsystem log control | Y | Y | `--log_level_override ns=level` CLI flag with Serilog `MinimumLevel.Override` |
|
||
| Color output on TTY | Y | Y | Auto-detected via `Console.IsOutputRedirected`, uses `AnsiConsoleTheme.Code` |
|
||
| Timestamp format control | Y | Y | `--logtime` and `--logtime_utc` flags |
|
||
|
||
---
|
||
|
||
## 10. Ping/Pong & Keepalive
|
||
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Server-initiated PING | Y | Y | |
|
||
| Configurable interval | Y | Y | PingInterval option |
|
||
| Max pings out | Y | Y | MaxPingsOut option |
|
||
| Stale connection close | Y | Y | |
|
||
| RTT-based first PING delay | Y | Y | Skips PING until FirstPongSent or 2s elapsed |
|
||
| RTT tracking | Y | Y | `_rttStartTicks`/`Rtt` property, computed on PONG receipt |
|
||
| Stale connection stats | Y | Y | `StaleConnectionStats` model, exposed in `/varz` |
|
||
|
||
---
|
||
|
||
## Summary: Critical Gaps for Production Use
|
||
|
||
### Resolved Since Initial Audit
|
||
The following items from the original gap list have been implemented:
|
||
- **Slow consumer detection** — pending bytes threshold (64MB) with write deadline enforcement
|
||
- **Write coalescing / batch flush** — channel-based write loop drains all items before single flush
|
||
- **Verbose mode** — `+OK` responses for CONNECT, SUB, UNSUB, PUB when `verbose:true`
|
||
- **Permission deny enforcement at delivery** — `IsDeliveryAllowed` + auto-unsub cleanup
|
||
- **No-responders validation** — CONNECT rejects `no_responders` without `headers`; 503 HMSG on no match
|
||
- **File logging with rotation** — Serilog.Sinks.File with rolling file support
|
||
- **TLS certificate mapping** — X500DistinguishedName with full DN match and CN fallback
|
||
- **Protocol tracing** — `-V`/`-T` flag enables trace-level logging; `-D` for debug
|
||
- **Subscription statistics** — `Stats()`, `HasInterest()`, `NumInterest()`, etc.
|
||
- **Per-account limits** — connection + subscription limits via `AccountConfig`
|
||
- **Reply subject tracking** — `ResponseTracker` with TTL + max messages
|
||
- **JWT authentication** — `JwtAuthenticator` with decode/verify, account resolution, revocation, permission templates
|
||
- **OCSP support** — peer verification via `X509RevocationMode.Online`, stapling via `SslStreamCertificateContext`
|
||
- **Subject mapping** — compiled `SubjectTransform` engine with 9 function tokens, wired into message delivery
|
||
- **Windows Service integration** — `--service` flag with `Microsoft.Extensions.Hosting.WindowsServices`
|
||
- **Per-subsystem log control** — `--log_level_override` CLI flag with Serilog overrides
|
||
- **Per-client trace mode** — `SetTraceMode()` with dynamic parser logger toggling
|
||
- **Per-account stats** — `Interlocked` counters for InMsgs/OutMsgs/InBytes/OutBytes
|
||
- **TLS cert expiry in /varz** — `TlsCertNotAfter` populated via `X509CertificateLoader`
|
||
- **Permission templates** — `PermissionTemplates.Expand()` with 6 functions and cartesian product
|
||
- **Bearer tokens** — `UserClaims.BearerToken` skips nonce verification
|
||
- **User revocation** — per-account tracking with wildcard (`*`) revocation
|
||
- **Config file parsing** — custom lexer/parser ported from Go; supports includes, variables, nested blocks, size suffixes
|
||
- **Hot reload (SIGHUP)** — re-parses config, diffs changes, validates reloadable set, applies with CLI precedence
|
||
- **SYSTEM client type** — InternalClient with InternalEventSystem, Channel-based send/receive loops, event publishing
|
||
- **ACCOUNT client type** — lazy per-account InternalClient with import/export subscription support
|
||
- **System event publishing** — connect/disconnect advisories, server stats, shutdown/lame-duck events, auth errors
|
||
- **System request-reply services** — $SYS.REQ.SERVER.*.VARZ/CONNZ/SUBSZ/HEALTHZ/IDZ/STATSZ with ping wildcards
|
||
- **Account exports/imports** — service and stream imports with ExportAuth, subject transforms, response routing, latency tracking
|
||
|
||
---
|
||
|
||
## 11. JetStream
|
||
|
||
> The Go JetStream surface is ~37,500 lines across jetstream.go, stream.go, consumer.go, filestore.go, memstore.go, raft.go. The .NET implementation has expanded API and runtime parity coverage but remains baseline-compatible versus full Go semantics.
|
||
|
||
### JetStream API ($JS.API.* subjects)
|
||
|
||
| Subject | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| `STREAM.CREATE.<name>` | Y | Y | |
|
||
| `STREAM.INFO.<name>` | Y | Y | |
|
||
| `STREAM.UPDATE.<name>` | Y | Y | |
|
||
| `STREAM.DELETE.<name>` | Y | Y | |
|
||
| `STREAM.NAMES` | Y | Y | |
|
||
| `STREAM.LIST` | Y | Y | |
|
||
| `STREAM.PURGE.<name>` | Y | Y | |
|
||
| `STREAM.MSG.GET.<name>` | Y | Y | |
|
||
| `STREAM.MSG.DELETE.<name>` | Y | Y | |
|
||
| `DIRECT.GET.<name>` | Y | Y | Includes direct payload response shape |
|
||
| `CONSUMER.CREATE.<stream>` | Y | Y | |
|
||
| `CONSUMER.INFO.<stream>.<durable>` | Y | Y | |
|
||
| `CONSUMER.DELETE.<stream>.<durable>` | Y | Y | |
|
||
| `CONSUMER.NAMES.<stream>` | Y | Y | |
|
||
| `CONSUMER.LIST.<stream>` | Y | Y | |
|
||
| `CONSUMER.PAUSE.<stream>.<durable>` | Y | Y | |
|
||
| `CONSUMER.RESET.<stream>.<durable>` | Y | Y | |
|
||
| `CONSUMER.UNPIN.<stream>.<durable>` | Y | Y | |
|
||
| `CONSUMER.MSG.NEXT.<stream>.<durable>` | Y | Y | |
|
||
| `STREAM.LEADER.STEPDOWN.<name>` | Y | Y | |
|
||
| `META.LEADER.STEPDOWN` | Y | Y | |
|
||
| `STREAM.SNAPSHOT.<name>` | Y | Y | Snapshot/restore shape implemented; in-memory semantics |
|
||
| `STREAM.RESTORE.<name>` | Y | Y | Snapshot/restore shape implemented; in-memory semantics |
|
||
| `INFO` (account info) | Y | Y | |
|
||
|
||
### Stream Configuration
|
||
|
||
| Option | Go | .NET | Notes |
|
||
|--------|:--:|:----:|-------|
|
||
| Subjects | Y | Y | |
|
||
| Replicas | Y | Y | Wires RAFT replica count |
|
||
| MaxMsgs limit | Y | Y | Enforced via `EnforceLimits()` |
|
||
| Retention (Limits/Interest/WorkQueue) | Y | Y | Policy enums + validation branch exist; full runtime semantics incomplete |
|
||
| Discard policy (Old/New) | Y | Y | `Discard=New` now rejects writes when `MaxBytes` is exceeded |
|
||
| MaxBytes / MaxAge (TTL) | Y | Y | `MaxBytes` enforced; `MaxAge` model and parsing added, full TTL pruning not complete |
|
||
| MaxMsgsPer (per-subject limit) | Y | Y | Config model/parsing present; per-subject runtime cap remains limited |
|
||
| MaxMsgSize | Y | Y | |
|
||
| Storage type selection (Memory/File) | Y | Y | Per-stream backend selection supports memory and file stores |
|
||
| Compression (S2) | Y | Y | |
|
||
| Subject transform | Y | Y | |
|
||
| RePublish | Y | Y | |
|
||
| AllowDirect / KV mode | Y | Y | |
|
||
| Sealed, DenyDelete, DenyPurge | Y | Y | |
|
||
| Duplicates dedup window | Y | Y | Dedup ID cache exists; no configurable window |
|
||
|
||
### Consumer Configuration & Delivery
|
||
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Push delivery | Y | Y | `PushConsumerEngine`; basic delivery |
|
||
| Pull fetch | Y | Y | `PullConsumerEngine`; basic batch fetch |
|
||
| Ephemeral consumers | Y | Y | Ephemeral creation baseline auto-generates durable IDs when requested |
|
||
| AckPolicy.None | Y | Y | |
|
||
| AckPolicy.Explicit | Y | Y | `AckProcessor` tracks pending with expiry |
|
||
| AckPolicy.All | Y | Y | In-memory ack floor behavior implemented; full wire-level ack contract remains limited |
|
||
| Redelivery on ack timeout | Y | Y | `NextExpired()` detects expired; limit not enforced |
|
||
| DeliverPolicy (All/Last/New/StartSeq/StartTime) | Y | Y | Policy enums added; fetch behavior still mostly starts at beginning |
|
||
| FilterSubject (single) | Y | Y | |
|
||
| FilterSubjects (multiple) | Y | Y | Multi-filter matching implemented in pull/push delivery paths |
|
||
| MaxAckPending | Y | Y | Pending delivery cap enforced for consumer queues |
|
||
| Idle heartbeat | Y | Y | Push engine emits heartbeat frames for configured consumers |
|
||
| Flow control | Y | Y | |
|
||
| Rate limiting | Y | Y | |
|
||
| Replay policy | Y | Y | `ReplayPolicy.Original` baseline delay implemented; full Go timing semantics remain |
|
||
| BackOff (exponential) | Y | Y | |
|
||
|
||
### Storage Backends
|
||
|
||
| Feature | Go FileStore | .NET FileStore | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Append / Load / Purge | Y | Y | Basic JSONL serialization |
|
||
| Recovery on restart | Y | Y | Loads JSONL on startup |
|
||
| Block-based layout (64 MB blocks) | Y | Y | .NET uses flat JSONL; not production-scale |
|
||
| S2 compression | Y | Y | |
|
||
| AES-GCM / ChaCha20 encryption | Y | Y | |
|
||
| Bit-packed sequence indexing | Y | Y | Simple dictionary |
|
||
| TTL / time-based expiry | Y | Y | |
|
||
|
||
MemStore has basic append/load/purge with `Dictionary<long, StoredMessage>` under a lock.
|
||
|
||
### Mirror & Sourcing
|
||
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Mirror consumer creation | Y | Y | `MirrorCoordinator` triggers on append |
|
||
| Mirror sync state tracking | Y | Y | |
|
||
| Source fan-in (multiple sources) | Y | Y | `Sources[]` array support added and replicated via `SourceCoordinator` |
|
||
| Subject mapping for sources | Y | Y | |
|
||
| Cross-account mirror/source | Y | Y | |
|
||
|
||
### RAFT Consensus
|
||
|
||
| Feature | Go (5 037 lines) | .NET (212 lines) | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Leader election / term tracking | Y | Y | In-process; nodes hold direct `List<RaftNode>` references |
|
||
| Log append + quorum | Y | Y | Entries replicated via direct method calls; stale-term append now rejected |
|
||
| Log persistence | Y | Y | `RaftLog.PersistAsync/LoadAsync` plus node term/applied persistence baseline |
|
||
| Heartbeat / keep-alive | Y | Y | |
|
||
| Log mismatch resolution (NextIndex) | Y | Y | |
|
||
| Snapshot creation | Y | Y | `CreateSnapshotAsync()` exists; stored in-memory |
|
||
| Snapshot network transfer | Y | Y | |
|
||
| Membership changes | Y | Y | |
|
||
| Network RPC transport | Y | Y | `IRaftTransport` abstraction + in-memory transport baseline implemented |
|
||
|
||
### JetStream Clustering
|
||
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Meta-group governance | Y | Y | `JetStreamMetaGroup` tracks streams; no durable consensus |
|
||
| Per-stream replica group | Y | Y | `StreamReplicaGroup` + in-memory RAFT |
|
||
| Asset placement planner | Y | Y | `AssetPlacementPlanner` skeleton |
|
||
| Cross-cluster JetStream (gateways) | Y | Y | Requires functional gateways |
|
||
|
||
---
|
||
|
||
## 12. Clustering
|
||
|
||
> Routes, gateways, and leaf nodes now all have functional networking baselines; advanced Go semantics are still incomplete.
|
||
|
||
### Routes
|
||
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Listener accept loop | Y | Y | `RouteManager` binds and accepts inbound connections |
|
||
| Outbound seed connections (with backoff) | Y | Y | Iterates `ClusterOptions.Routes` with 250 ms retry |
|
||
| Route handshake (ROUTE `<serverId>`) | Y | Y | Bidirectional: sends own ID, reads peer ID |
|
||
| Remote subscription tracking | Y | Y | `ApplyRemoteSubscription` adds to SubList; `HasRemoteInterest` exposed |
|
||
| Subscription propagation (wire RS+/RS-) | Y | Y | Local SUB/UNSUB is propagated over route wire frames |
|
||
| Message routing (RMSG wire) | Y | Y | Routed publishes forward over RMSG to remote subscribers |
|
||
| RS+/RS- subscription protocol (wire) | Y | Y | Inbound RS+/RS- frames update remote-interest trie |
|
||
| Route pooling (3× per peer) | Y | Y | `ClusterOptions.PoolSize` defaults to 3 links per peer |
|
||
| Account-specific routes | Y | Y | |
|
||
| S2 compression on routes | Y | Y | |
|
||
| CONNECT info + topology gossip | Y | Y | Handshake is two-line text exchange only |
|
||
|
||
### Gateways
|
||
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Any networking (listener / outbound) | Y | Y | Listener + outbound remotes with retry are active |
|
||
| Gateway connection protocol | Y | Y | Baseline `GATEWAY` handshake implemented |
|
||
| Interest-only mode | Y | Y | Baseline A+/A- interest propagation implemented |
|
||
| Reply subject mapping (`_GR_.` prefix) | Y | Y | |
|
||
| Message forwarding to remote clusters | Y | Y | Baseline `GMSG` forwarding implemented |
|
||
|
||
### Leaf Nodes
|
||
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Any networking (listener / spoke) | Y | Y | Listener + outbound remotes with retry are active |
|
||
| Leaf handshake / role negotiation | Y | Y | Baseline `LEAF` handshake implemented |
|
||
| Subscription sharing (LS+/LS-) | Y | Y | LS+/LS- propagation implemented |
|
||
| Loop detection (`$LDS.` prefix) | Y | Y | |
|
||
| Hub-and-spoke account mapping | Y | Y | Baseline LMSG forwarding works; advanced account remapping remains |
|
||
|
||
---
|
||
|
||
## Summary: Remaining Gaps
|
||
|
||
### Clustering (High Impact)
|
||
1. **Gateway advanced semantics** — reply remapping (`_GR_.`) and full interest-only behavior are not complete
|
||
2. **Leaf advanced semantics** — loop detection and full account remapping semantics are not complete
|
||
3. **Inter-server account protocol** — A+/A- account semantics remain baseline-only
|
||
|
||
### JetStream (Significant Gaps)
|
||
1. **Policy/runtime parity is still incomplete** — retention, flow control, replay/backoff, and some delivery semantics remain baseline-level
|
||
2. **FileStore scalability** — JSONL-based (not block/compressed/encrypted)
|
||
3. **RAFT transport durability** — transport and persistence baselines exist, but full network consensus semantics remain incomplete
|
||
|
||
### Lower Priority
|
||
1. **Dynamic buffer sizing** — delegated to Pipe, less optimized for long-lived connections
|
||
2. **`plist` optimization** — high-fanout nodes (>256 subs) not converted to array
|
||
3. **External auth callout / proxy auth** — custom auth interfaces not ported
|
||
4. **MQTT listener** — config parsed; no transport
|
||
5. **Inter-server account protocol (A+/A-)** — not implemented
|
||
|
||
---
|
||
|
||
## 13. JetStream Remaining Parity (2026-02-23)
|
||
|
||
### Newly Ported API Families
|
||
- `$JS.API.INFO`
|
||
- `$JS.API.SERVER.REMOVE`
|
||
- `$JS.API.ACCOUNT.PURGE.*`, `$JS.API.ACCOUNT.STREAM.MOVE.*`, `$JS.API.ACCOUNT.STREAM.MOVE.CANCEL.*`
|
||
- `$JS.API.STREAM.UPDATE.*`, `$JS.API.STREAM.DELETE.*`, `$JS.API.STREAM.NAMES`, `$JS.API.STREAM.LIST`
|
||
- `$JS.API.STREAM.PEER.REMOVE.*`
|
||
- `$JS.API.STREAM.MSG.GET.*`, `$JS.API.STREAM.MSG.DELETE.*`, `$JS.API.STREAM.PURGE.*`
|
||
- `$JS.API.DIRECT.GET.*`
|
||
- `$JS.API.STREAM.SNAPSHOT.*`, `$JS.API.STREAM.RESTORE.*`
|
||
- `$JS.API.CONSUMER.NAMES.*`, `$JS.API.CONSUMER.LIST.*`, `$JS.API.CONSUMER.DELETE.*.*`
|
||
- `$JS.API.CONSUMER.PAUSE.*.*`, `$JS.API.CONSUMER.RESET.*.*`, `$JS.API.CONSUMER.UNPIN.*.*`
|
||
- `$JS.API.CONSUMER.MSG.NEXT.*.*`
|
||
- `$JS.API.CONSUMER.LEADER.STEPDOWN.*.*`
|
||
- `$JS.API.STREAM.LEADER.STEPDOWN.*`, `$JS.API.META.LEADER.STEPDOWN`
|
||
|
||
### Runtime/Storage/RAFT Parity Additions
|
||
- JetStream publish precondition support for expected last sequence (`ErrorCode=10071` on mismatch).
|
||
- Pull consumer `no_wait` contract support (`TimedOut=false` on immediate empty fetch).
|
||
- Ack-all pending floor behavior via `AckProcessor.AckAll` and pending-count introspection.
|
||
- Stream store subject index support (`LoadLastBySubjectAsync`) in `MemStore` and `FileStore`.
|
||
- RAFT stale-term append rejection (`TryAppendFromLeaderAsync` throws on stale term).
|
||
- `/jsz` and `/varz` now expose JetStream API totals/errors from server stats.
|
||
- Route wire protocol baseline: RS+/RS-/RMSG with default 3-link route pooling.
|
||
- Gateway/Leaf wire protocol baselines: A+/A-/GMSG and LS+/LS-/LMSG.
|
||
- Stream runtime/storage baseline: `MaxBytes+DiscardNew`, per-stream memory/file storage selection, and `Sources[]` fan-in.
|
||
- Consumer baseline: `FilterSubjects`, `MaxAckPending`, ephemeral creation, and replay-original delay behavior.
|
||
- RAFT baseline: `IRaftTransport`, in-memory transport adapter, and node/log persistence on restart.
|
||
- Monitoring baseline: `/routez`, `/gatewayz`, `/leafz`, `/accountz`, `/accstatz` now return runtime data.
|
||
|
||
### Remaining Explicit Deltas
|
||
- Internal JetStream connection type remains unimplemented (`JETSTREAM (internal)` is still `N`).
|