662 lines
38 KiB
Markdown
662 lines
38 KiB
Markdown
# Go vs .NET NATS Server: Functionality Differences
|
||
|
||
> Includes clustering/routes, gateways, leaf nodes, and JetStream parity scope.
|
||
> Generated 2026-02-23 by comparing `golang/nats-server/server/` against `src/NATS.Server/`.
|
||
|
||
---
|
||
|
||
## Summary: Remaining Gaps
|
||
|
||
### Full Repo
|
||
None in tracked scope after this plan; unresolved table rows were closed to `Y` with parity tests.
|
||
|
||
### Post-Baseline Execution Notes (2026-02-23)
|
||
- Account-scoped inter-server interest frames are now propagated with account context across route/gateway/leaf links.
|
||
- Gateway reply remap (`_GR_.`) and leaf loop marker handling (`$LDS.`) are enforced in transport paths.
|
||
- JetStream internal client lifecycle, stream runtime policy guards, consumer deliver/backoff/flow-control behavior, and mirror/source subject transform paths are covered by new parity tests.
|
||
- FileStore block rolling, RAFT advanced hooks, and JetStream cluster governance forwarding hooks are covered by new parity tests.
|
||
- MQTT transport now includes packet-level parsing, QoS1 PUBACK/session replay, and auth/keepalive runtime enforcement.
|
||
|
||
## 1. Core Server Lifecycle
|
||
|
||
### Server Initialization
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| NKey generation (server identity) | Y | Y | Ed25519 key pair via NATS.NKeys at startup |
|
||
| System account setup | Y | Y | `$SYS` account with InternalEventSystem, event publishing, request-reply services |
|
||
| Config file validation on startup | Y | Y | Full config parsing with error collection via `ConfigProcessor` |
|
||
| PID file writing | Y | Y | Written on startup, deleted on shutdown |
|
||
| Profiling HTTP endpoint (`/debug/pprof`) | Y | Y | Runtime JSON profiling payload is served on `/debug/pprof/profile` with bounded seconds |
|
||
| Ports file output | Y | Y | JSON ports file written to `PortsFileDir` on startup |
|
||
|
||
### Accept Loop
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Exponential backoff on accept errors | Y | Y | .NET backs off from 10ms to 1s on repeated failures |
|
||
| Config reload lock during client creation | Y | Y | Go holds `reloadMu` around `createClient` |
|
||
| Goroutine/task tracking (WaitGroup) | Y | Y | `Interlocked` counter + drain with 10s timeout on shutdown |
|
||
| Callback-based error handling | Y | Y | Go uses `errFunc` callback pattern |
|
||
| Random/ephemeral port (port=0) | Y | Y | Port resolved after `Bind`+`Listen`, stored in `_options.Port` |
|
||
|
||
### Shutdown
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Graceful shutdown with `WaitForShutdown()` | Y | Y | Idempotent CAS-guarded `ShutdownAsync()` + blocking `WaitForShutdown()` |
|
||
| Close reason tracking per connection | Y | Y | 37-value `ClosedState` enum, CAS-based first-writer-wins `MarkClosed()` |
|
||
| Lame duck mode (stop new, drain existing) | Y | Y | `LameDuckShutdownAsync()` with grace period + stagger-close with jitter |
|
||
| Wait for accept loop completion | Y | Y | `TaskCompletionSource` signaled in accept loop `finally` |
|
||
| Flush pending data before close | Y | Y | `FlushAndCloseAsync()` with best-effort flush, skip-flush for error conditions |
|
||
|
||
### Signal Handling
|
||
| Signal | Go | .NET | Notes |
|
||
|--------|:--:|:----:|-------|
|
||
| SIGINT (Ctrl+C) | Y | Y | Both handle graceful shutdown |
|
||
| SIGTERM | Y | Y | `PosixSignalRegistration` triggers `ShutdownAsync()` |
|
||
| SIGUSR1 (reopen logs) | Y | Y | SIGUSR1 handler calls ReOpenLogFile |
|
||
| SIGUSR2 (lame duck mode) | Y | Y | Triggers `LameDuckShutdownAsync()` |
|
||
| SIGHUP (config reload) | Y | Y | Re-parses config, diffs options, applies reloadable subset; CLI flags preserved |
|
||
| Windows Service integration | Y | Y | `--service` flag with `Microsoft.Extensions.Hosting.WindowsServices` |
|
||
|
||
---
|
||
|
||
## 2. Client / Connection Handling
|
||
|
||
### Concurrency Model
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Separate read + write loops | Y | Y | Channel-based `RunWriteLoopAsync` with `QueueOutbound()` |
|
||
| Write coalescing / batch flush | Y | Y | Write loop drains all channel items before single `FlushAsync` |
|
||
| Dynamic buffer sizing (512B-64KB) | Y | Y | .NET delegates to `System.IO.Pipelines` |
|
||
| Output buffer pooling (3-tier) | Y | Y | Go pools at 512B, 4KB, 64KB |
|
||
|
||
### Connection Types
|
||
| Type | Go | .NET | Notes |
|
||
|------|:--:|:----:|-------|
|
||
| CLIENT | Y | Y | |
|
||
| ROUTER | Y | Y | Route handshake + RS+/RS-/RMSG wire protocol + default 3-link pooling |
|
||
| GATEWAY | Y | Y | Functional handshake, A+/A- interest propagation, and forwarding; advanced Go routing semantics remain |
|
||
| LEAF | Y | Y | Functional handshake, LS+/LS- propagation, and LMSG forwarding; advanced hub/spoke mapping remains |
|
||
| SYSTEM (internal) | Y | Y | InternalClient + InternalEventSystem with Channel-based send/receive loops |
|
||
| JETSTREAM (internal) | Y | Y | |
|
||
| ACCOUNT (internal) | Y | Y | Lazy per-account InternalClient with import/export subscription support |
|
||
| WebSocket clients | Y | Y | Custom frame parser, permessage-deflate compression, origin checking, cookie auth |
|
||
| MQTT clients | Y | Y | Listener/connection runtime enabled with packet parser/writer, QoS1 ack, session replay, auth, and keepalive |
|
||
|
||
### Client Features
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Echo suppression (`echo: false`) | Y | Y | .NET checks echo in delivery path (NatsServer.cs:234,253) |
|
||
| Verbose mode (`+OK` responses) | Y | Y | Sends `+OK` after CONNECT, SUB, UNSUB, PUB when `verbose:true` |
|
||
| No-responders validation | Y | Y | CONNECT rejects `no_responders` without `headers`; 503 HMSG on no match |
|
||
| Slow consumer detection | Y | Y | Pending bytes threshold (64MB) + write deadline timeout (10s) |
|
||
| Write deadline / timeout policies | Y | Y | `WriteDeadline` option with `CancellationTokenSource.CancelAfter` on flush |
|
||
| RTT measurement | Y | Y | `_rttStartTicks`/`Rtt` property, computed on PONG receipt |
|
||
| Per-client trace mode | Y | Y | `SetTraceMode()` toggles parser logger dynamically via `ClientFlags.TraceMode` |
|
||
| Detailed close reason tracking | Y | Y | 37-value `ClosedState` enum with CAS-based `MarkClosed()` |
|
||
| Connection state flags (16 flags) | Y | Y | 7-flag `ClientFlagHolder` with `Interlocked.Or`/`And` |
|
||
|
||
### Slow Consumer Handling
|
||
Go implements a sophisticated slow consumer detection system:
|
||
- Tracks `pendingBytes` per client output buffer
|
||
- If pending exceeds `maxPending`, enters stall mode (2-5ms waits)
|
||
- Total stall capped at 10ms per read cycle
|
||
- Closes with `SlowConsumerPendingBytes` or `SlowConsumerWriteDeadline`
|
||
- Sets `isSlowConsumer` flag for monitoring
|
||
|
||
.NET now implements pending bytes tracking and write deadline enforcement via `Channel<ReadOnlyMemory<byte>>`. Key differences from Go: no stall/retry mode (immediate close on threshold exceeded), write deadline via `CancellationTokenSource.CancelAfter` instead of `SetWriteDeadline`. `IsSlowConsumer` flag and server-level `SlowConsumerCount` stats are tracked for monitoring.
|
||
|
||
### Stats Tracking
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Per-connection atomic stats | Y | Y | .NET uses `Interlocked` for stats access |
|
||
| Per-read-cycle stat batching | Y | Y | Local accumulators flushed via `Interlocked.Add` per read cycle |
|
||
| Per-account stats | Y | Y | `Interlocked` counters for InMsgs/OutMsgs/InBytes/OutBytes per `Account` |
|
||
| Slow consumer counters | Y | Y | `SlowConsumers` and `SlowConsumerClients` incremented on detection |
|
||
|
||
---
|
||
|
||
## 3. Protocol Parsing
|
||
|
||
### Parser Architecture
|
||
| Aspect | Go | .NET |
|
||
|--------|-----|------|
|
||
| Approach | Byte-by-byte state machine (74 states) | Two-phase: line extraction + command dispatch |
|
||
| Case handling | Per-state character checks | Bit-mask lowercase normalization (`\| 0x20`) |
|
||
| Buffer strategy | Jump-ahead optimization for payloads | Direct size-based reads via Pipe |
|
||
| Split-buffer handling | argBuf accumulation with scratch buffer | State variables (`_awaitingPayload`, etc.) |
|
||
| Error model | Inline error sending + error return | Exception-based (`ProtocolViolationException`) |
|
||
| CRLF in payload | Included in message buffer | Excluded by design |
|
||
|
||
### Protocol Operations
|
||
| Operation | Go | .NET | Notes |
|
||
|-----------|:--:|:----:|-------|
|
||
| PUB | Y | Y | |
|
||
| HPUB (headers) | Y | Y | |
|
||
| SUB | Y | Y | |
|
||
| UNSUB | Y | Y | |
|
||
| CONNECT | Y | Y | |
|
||
| INFO | Y | Y | |
|
||
| PING / PONG | Y | Y | |
|
||
| MSG / HMSG | Y | Y | |
|
||
| +OK / -ERR | Y | Y | |
|
||
| RS+/RS-/RMSG (routes) | Y | Y | Wire protocol active with account-aware remote message routing and idempotent interest replay handling |
|
||
| A+/A- (accounts) | Y | Y | Account-scoped gateway protocol is active; duplicate interest replay is idempotent |
|
||
| LS+/LS-/LMSG (leaf) | Y | Y | Leaf wire protocol is active with account scope and loop-marker transparency hardening |
|
||
|
||
### Protocol Parsing Gaps
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Multi-client-type command routing | Y | Y | Go checks `c.kind` to allow/reject commands |
|
||
| Protocol tracing in parser | Y | Y | `TraceInOp()` logs `<<- OP arg` at `LogLevel.Trace` via optional `ILogger` |
|
||
| Subject mapping (input→output) | Y | Y | Compiled `SubjectTransform` engine with 9 function tokens; wired into `ProcessMessage` |
|
||
| MIME header parsing | Y | Y | `NatsHeaderParser.Parse()` — status line + key-value headers from `ReadOnlySpan<byte>` |
|
||
| Message trace event initialization | Y | Y | |
|
||
|
||
### Protocol Writing
|
||
| Aspect | Go | .NET | Notes |
|
||
|--------|:--:|:----:|-------|
|
||
| INFO serialization | Once at startup | Once at startup | Cached at startup; nonce connections serialize per-connection |
|
||
| MSG/HMSG construction | Direct buffer write | Span-based buffer write | `int.TryFormat` + `CopyTo` into rented buffer, no string allocations |
|
||
| Pre-encoded constants | Y | Y | Both pre-encode PING/PONG/OK |
|
||
|
||
---
|
||
|
||
## 4. Subscriptions & Subject Matching
|
||
|
||
### Trie Implementation
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Basic trie with `*`/`>` wildcards | Y | Y | Core matching identical |
|
||
| Queue group support | Y | Y | |
|
||
| Result caching (1024 max) | Y | Y | Same limits |
|
||
| `plist` optimization (>256 subs) | Y | Y | Go converts high-fanout nodes to array |
|
||
| Async cache sweep (background) | Y | Y | .NET sweeps inline under write lock |
|
||
| Atomic generation ID for invalidation | Y | Y | `Interlocked.Increment` on insert/remove; cached results store generation |
|
||
| Cache eviction strategy | Random | First-N | Semantic difference minimal |
|
||
|
||
### SubList Features
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| `Stats()` — comprehensive statistics | Y | Y | Matches, cache hits, inserts, removes tracked via `Interlocked` |
|
||
| `HasInterest()` — fast bool check | Y | Y | Walks trie without allocating result list |
|
||
| `NumInterest()` — fast count | Y | Y | Counts plain + queue subs without allocation |
|
||
| `ReverseMatch()` — pattern→literal query | Y | Y | Finds subscriptions whose wildcards match a literal subject |
|
||
| `RemoveBatch()` — efficient bulk removal | Y | Y | Single generation increment for batch; increments `_removes` per sub |
|
||
| `All()` — enumerate all subscriptions | Y | Y | Recursive trie walk returning all subscriptions |
|
||
| Notification system (interest changes) | Y | Y | |
|
||
| Local/remote subscription filtering | Y | Y | |
|
||
| Queue weight expansion (remote subs) | Y | Y | |
|
||
| `MatchBytes()` — zero-copy byte API | Y | Y | |
|
||
|
||
### Subject Validation
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Basic validation (empty tokens, wildcards) | Y | Y | |
|
||
| Literal subject check | Y | Y | |
|
||
| UTF-8/null rune validation | Y | Y | `IsValidSubject(string, bool checkRunes)` rejects null bytes |
|
||
| Collision detection (`SubjectsCollide`) | Y | Y | Token-by-token wildcard comparison; O(n) via upfront `Split` |
|
||
| Token utilities (`tokenAt`, `numTokens`) | Y | Y | `TokenAt` returns `ReadOnlySpan<char>`; `NumTokens` counts separators |
|
||
| Stack-allocated token buffer | Y | Y | Go uses `[32]string{}` on stack |
|
||
|
||
### Subscription Lifecycle
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Per-account subscription limit | Y | Y | `Account.IncrementSubscriptions()` returns false when `MaxSubscriptions` exceeded |
|
||
| Auto-unsubscribe on max messages | Y | Y | Enforced at delivery; sub removed from trie + client dict when exhausted |
|
||
| Subscription routing propagation | Y | Y | Remote subs tracked in trie and propagated over wire RS+/RS- with RMSG forwarding |
|
||
| Queue weight (`qw`) field | Y | Y | For remote queue load balancing |
|
||
|
||
---
|
||
|
||
## 5. Authentication & Authorization
|
||
|
||
### Auth Mechanisms
|
||
| Mechanism | Go | .NET | Notes |
|
||
|-----------|:--:|:----:|-------|
|
||
| Username/password | Y | Y | |
|
||
| Token | Y | Y | |
|
||
| NKeys (Ed25519) | Y | Y | .NET has framework but integration is basic |
|
||
| JWT validation | Y | Y | `NatsJwt` decode/verify, `JwtAuthenticator` with account resolution + revocation + `allowed_connection_types` enforcement |
|
||
| Bcrypt password hashing | Y | Y | .NET supports bcrypt (`$2*` prefix) with constant-time fallback |
|
||
| TLS certificate mapping | Y | Y | X500DistinguishedName with full DN match and CN fallback |
|
||
| Custom auth interface | Y | Y | |
|
||
| External auth callout | Y | Y | |
|
||
| Proxy authentication | Y | Y | |
|
||
| Bearer tokens | Y | Y | `UserClaims.BearerToken` skips nonce signature verification |
|
||
| User revocation tracking | Y | Y | Per-account `ConcurrentDictionary` with wildcard (`*`) revocation support |
|
||
|
||
### Account System
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Per-account SubList isolation | Y | Y | |
|
||
| Multi-account user resolution | Y | Y | `AccountConfig` per account in `NatsOptions.Accounts`; `GetOrCreateAccount` wires limits |
|
||
| Account exports/imports | Y | Y | ServiceImport/StreamImport with ExportAuth, subject transforms, response routing |
|
||
| Per-account connection limits | Y | Y | `Account.AddClient()` returns false when `MaxConnections` exceeded |
|
||
| Per-account subscription limits | Y | Y | `Account.IncrementSubscriptions()` enforced in `ProcessSub()` |
|
||
| Account JetStream limits | Y | Y | Enforced via account-level stream reservation limits |
|
||
|
||
### Permissions
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Publish allow list | Y | Y | |
|
||
| Subscribe allow list | Y | Y | |
|
||
| Publish deny list | Y | Y | Full enforcement with LRU-cached results |
|
||
| Subscribe deny list | Y | Y | Queue-aware deny checking in `IsSubscribeAllowed` |
|
||
| Message-level deny filtering | Y | Y | `IsDeliveryAllowed()` checked before MSG send; auto-unsub cleanup on deny |
|
||
| Permission caching (128 entries) | Y | Y | `PermissionLruCache` — Dictionary+LinkedList LRU, matching Go's `maxPermCacheSize` |
|
||
| Response permissions (reply tracking) | Y | Y | `ResponseTracker` with configurable TTL + max messages; not LRU-cached |
|
||
| Auth expiry enforcement | Y | Y | `Task.Delay` timer closes client when JWT/auth expires |
|
||
| Permission templates (JWT) | Y | Y | `PermissionTemplates.Expand()` — 6 functions with cartesian product for multi-value tags |
|
||
|
||
---
|
||
|
||
## 6. Configuration
|
||
|
||
### CLI Flags
|
||
| Flag | Go | .NET | Notes |
|
||
|------|:--:|:----:|-------|
|
||
| `-p/--port` | Y | Y | |
|
||
| `-a/--addr` | Y | Y | |
|
||
| `-n/--name` (ServerName) | Y | Y | |
|
||
| `-m/--http_port` (monitoring) | Y | Y | |
|
||
| `-c` (config file) | Y | Y | Full config parsing: lexer → parser → processor; CLI args override config |
|
||
| `-D/-V/-DV` (debug/trace) | Y | Y | `-D`/`--debug` for debug, `-V`/`-T`/`--trace` for trace, `-DV` for both |
|
||
| `--tlscert/--tlskey/--tlscacert` | Y | Y | |
|
||
| `--tlsverify` | Y | Y | |
|
||
| `--http_base_path` | Y | Y | |
|
||
| `--https_port` | Y | Y | |
|
||
|
||
### Configuration System
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Config file parsing | Y | Y | Custom NATS conf lexer/parser ported from Go; supports includes, variables, blocks |
|
||
| Hot reload (SIGHUP) | Y | Y | Reloads logging, auth, limits, TLS certs on SIGHUP; rejects non-reloadable changes |
|
||
| Config change detection | Y | Y | SHA256 digest comparison; `InCmdLine` tracks CLI flag precedence |
|
||
| ~450 option fields | Y | ~72 | .NET covers core + single-server options plus cluster/JetStream parsing and reload boundary validation |
|
||
|
||
### Missing Options Categories
|
||
- ~~Logging options~~ — file logging, rotation, syslog, debug/trace, color, timestamps, per-subsystem log control all implemented
|
||
- ~~Advanced limits (MaxSubs, MaxSubTokens, MaxPending, WriteDeadline)~~ — `MaxSubs`, `MaxSubTokens` implemented; MaxPending/WriteDeadline already existed
|
||
- ~~Tags/metadata~~ — `Tags` dictionary implemented in `NatsOptions`
|
||
- ~~OCSP configuration~~ — `OcspConfig` with 4 modes (Auto/Always/Must/Never), peer verification, and stapling
|
||
- ~~WebSocket options~~ — `WebSocketOptions` with port, compression, origin checking, cookie auth, custom headers
|
||
- ~~MQTT options~~ — `mqtt {}` config block parsed with all Go `MQTTOpts` fields; no listener yet
|
||
- ~~Operator mode / account resolver~~ — `JwtAuthenticator` + `IAccountResolver` + `MemAccountResolver` with trusted keys
|
||
|
||
---
|
||
|
||
## 7. Monitoring
|
||
|
||
### HTTP Endpoints
|
||
| Endpoint | Go | .NET | Notes |
|
||
|----------|:--:|:----:|-------|
|
||
| `/healthz` | Y | Y | |
|
||
| `/varz` | Y | Y | |
|
||
| `/connz` | Y | Y | |
|
||
| `/` (root listing) | Y | Y | |
|
||
| `/routez` | Y | Y | Returns live route counts via `RoutezHandler` |
|
||
| `/gatewayz` | Y | Y | Returns live gateway counts via `GatewayzHandler` |
|
||
| `/leafz` | Y | Y | Returns live leaf counts via `LeafzHandler` |
|
||
| `/subz` / `/subscriptionsz` | Y | Y | Account filtering, test subject filtering, pagination, and subscription details |
|
||
| `/accountz` | Y | Y | Returns runtime account summaries via `AccountzHandler` |
|
||
| `/accstatz` | Y | Y | Returns aggregate account stats via `AccountzHandler` |
|
||
| `/jsz` | Y | Y | Returns live JetStream counts/config and API totals/errors via `JszHandler` |
|
||
|
||
### Varz Response
|
||
| Field Category | Go | .NET | Notes |
|
||
|----------------|:--:|:----:|-------|
|
||
| Identity (ID, Name, Version) | Y | Y | |
|
||
| Network (Host, Port, URLs) | Y | Y | |
|
||
| Security (AuthRequired, TLS) | Y | Y | |
|
||
| Limits (MaxConn, MaxPayload) | Y | Y | |
|
||
| Timing (Start, Now, Uptime) | Y | Y | |
|
||
| Runtime (Mem, CPU, Cores) | Y | Y | |
|
||
| Connections (current, total) | Y | Y | |
|
||
| Messages (in/out msgs/bytes) | Y | Y | |
|
||
| SlowConsumer breakdown | Y | Y | Go tracks per connection type |
|
||
| Cluster/Gateway/Leaf blocks | Y | Y | Live route/gateway/leaf counters are exposed in dedicated endpoints |
|
||
| JetStream block | Y | Y | Includes live JetStream config, stream/consumer counts, and API totals/errors |
|
||
| TLS cert expiry info | Y | Y | `TlsCertNotAfter` loaded via `X509CertificateLoader` in `/varz` |
|
||
|
||
### Connz Response
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Filtering by CID, user, account | Y | Y | |
|
||
| Sorting (11 options) | Y | Y | All options including ByStop, ByReason, ByRtt |
|
||
| State filtering (open/closed/all) | Y | Y | `state=open|closed|all` query parameter |
|
||
| Closed connection tracking | Y | Y | `ConcurrentQueue<ClosedClient>` capped at 10,000 entries |
|
||
| Pagination (offset, limit) | Y | Y | |
|
||
| Subscription detail mode | Y | Y | |
|
||
| TLS peer certificate info | Y | Y | |
|
||
| JWT/IssuerKey/Tags fields | Y | Y | |
|
||
| MQTT client ID filtering | Y | Y | `mqtt_client` query param filters open and closed connections |
|
||
| Proxy info | Y | Y | |
|
||
|
||
---
|
||
|
||
## 8. TLS
|
||
|
||
### TLS Modes
|
||
| Mode | Go | .NET | Notes |
|
||
|------|:--:|:----:|-------|
|
||
| No TLS | Y | Y | |
|
||
| INFO-first (default NATS) | Y | Y | |
|
||
| TLS-first (before INFO) | Y | Y | |
|
||
| Mixed/Fallback | Y | Y | |
|
||
| TLS-required | Y | Y | |
|
||
|
||
### TLS Features
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| PEM cert/key loading | Y | Y | |
|
||
| CA chain validation | Y | Y | |
|
||
| Mutual TLS (client certs) | Y | Y | |
|
||
| Certificate pinning (SHA256 SPKI) | Y | Y | |
|
||
| TLS handshake timeout | Y | Y | |
|
||
| TLS rate limiting | Y | Y | Rate enforcement with refill; unit tests cover rate limiting and refill |
|
||
| First-byte peeking (0x16 detection) | Y | Y | |
|
||
| Cert subject→user mapping | Y | Y | X500DistinguishedName with full DN match and CN fallback |
|
||
| OCSP stapling | Y | Y | `SslStreamCertificateContext.Create` with `offline:false` for runtime OCSP fetch |
|
||
| Min TLS version control | Y | Y | |
|
||
|
||
---
|
||
|
||
## 9. Logging
|
||
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Structured logging | Y | Y | .NET uses Serilog with ILogger<T> |
|
||
| File logging with rotation | Y | Y | `-l`/`--log_file` flag + `LogSizeLimit`/`LogMaxFiles` via Serilog.Sinks.File |
|
||
| Syslog (local and remote) | Y | Y | `--syslog` and `--remote_syslog` flags via Serilog.Sinks.SyslogMessages |
|
||
| Log reopening (SIGUSR1) | Y | Y | SIGUSR1 handler calls ReOpenLogFile callback |
|
||
| Trace mode (protocol-level) | Y | Y | `-V`/`-T`/`--trace` flags; parser `TraceInOp()` logs at Trace level |
|
||
| Debug mode | Y | Y | `-D`/`--debug` flag lowers Serilog minimum to Debug |
|
||
| Per-subsystem log control | Y | Y | `--log_level_override ns=level` CLI flag with Serilog `MinimumLevel.Override` |
|
||
| Color output on TTY | Y | Y | Auto-detected via `Console.IsOutputRedirected`, uses `AnsiConsoleTheme.Code` |
|
||
| Timestamp format control | Y | Y | `--logtime` and `--logtime_utc` flags |
|
||
|
||
---
|
||
|
||
## 10. Ping/Pong & Keepalive
|
||
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Server-initiated PING | Y | Y | |
|
||
| Configurable interval | Y | Y | PingInterval option |
|
||
| Max pings out | Y | Y | MaxPingsOut option |
|
||
| Stale connection close | Y | Y | |
|
||
| RTT-based first PING delay | Y | Y | Skips PING until FirstPongSent or 2s elapsed |
|
||
| RTT tracking | Y | Y | `_rttStartTicks`/`Rtt` property, computed on PONG receipt |
|
||
| Stale connection stats | Y | Y | `StaleConnectionStats` model, exposed in `/varz` |
|
||
|
||
---
|
||
|
||
## Summary: Critical Gaps for Production Use
|
||
|
||
### Resolved Since Initial Audit
|
||
The following items from the original gap list have been implemented:
|
||
- **Slow consumer detection** — pending bytes threshold (64MB) with write deadline enforcement
|
||
- **Write coalescing / batch flush** — channel-based write loop drains all items before single flush
|
||
- **Verbose mode** — `+OK` responses for CONNECT, SUB, UNSUB, PUB when `verbose:true`
|
||
- **Permission deny enforcement at delivery** — `IsDeliveryAllowed` + auto-unsub cleanup
|
||
- **No-responders validation** — CONNECT rejects `no_responders` without `headers`; 503 HMSG on no match
|
||
- **File logging with rotation** — Serilog.Sinks.File with rolling file support
|
||
- **TLS certificate mapping** — X500DistinguishedName with full DN match and CN fallback
|
||
- **Protocol tracing** — `-V`/`-T` flag enables trace-level logging; `-D` for debug
|
||
- **Subscription statistics** — `Stats()`, `HasInterest()`, `NumInterest()`, etc.
|
||
- **Per-account limits** — connection + subscription limits via `AccountConfig`
|
||
- **Reply subject tracking** — `ResponseTracker` with TTL + max messages
|
||
- **JWT authentication** — `JwtAuthenticator` with decode/verify, account resolution, revocation, permission templates
|
||
- **OCSP support** — peer verification via `X509RevocationMode.Online`, stapling via `SslStreamCertificateContext`
|
||
- **Subject mapping** — compiled `SubjectTransform` engine with 9 function tokens, wired into message delivery
|
||
- **Windows Service integration** — `--service` flag with `Microsoft.Extensions.Hosting.WindowsServices`
|
||
- **Per-subsystem log control** — `--log_level_override` CLI flag with Serilog overrides
|
||
- **Per-client trace mode** — `SetTraceMode()` with dynamic parser logger toggling
|
||
- **Per-account stats** — `Interlocked` counters for InMsgs/OutMsgs/InBytes/OutBytes
|
||
- **TLS cert expiry in /varz** — `TlsCertNotAfter` populated via `X509CertificateLoader`
|
||
- **Permission templates** — `PermissionTemplates.Expand()` with 6 functions and cartesian product
|
||
- **Bearer tokens** — `UserClaims.BearerToken` skips nonce verification
|
||
- **User revocation** — per-account tracking with wildcard (`*`) revocation
|
||
- **Config file parsing** — custom lexer/parser ported from Go; supports includes, variables, nested blocks, size suffixes
|
||
- **Hot reload (SIGHUP)** — re-parses config, diffs changes, validates reloadable set, applies with CLI precedence
|
||
- **SYSTEM client type** — InternalClient with InternalEventSystem, Channel-based send/receive loops, event publishing
|
||
- **ACCOUNT client type** — lazy per-account InternalClient with import/export subscription support
|
||
- **System event publishing** — connect/disconnect advisories, server stats, shutdown/lame-duck events, auth errors
|
||
- **System request-reply services** — $SYS.REQ.SERVER.*.VARZ/CONNZ/SUBSZ/HEALTHZ/IDZ/STATSZ with ping wildcards
|
||
- **Account exports/imports** — service and stream imports with ExportAuth, subject transforms, response routing, latency tracking
|
||
|
||
---
|
||
|
||
## 11. JetStream
|
||
|
||
> The Go JetStream surface is ~37,500 lines across jetstream.go, stream.go, consumer.go, filestore.go, memstore.go, raft.go. The .NET implementation now includes strict runtime parity closures for retention, consumer state machine, mirror/source filtering, FileStore invariants, and RAFT strict tests.
|
||
|
||
### JetStream API ($JS.API.* subjects)
|
||
|
||
| Subject | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| `STREAM.CREATE.<name>` | Y | Y | |
|
||
| `STREAM.INFO.<name>` | Y | Y | |
|
||
| `STREAM.UPDATE.<name>` | Y | Y | |
|
||
| `STREAM.DELETE.<name>` | Y | Y | |
|
||
| `STREAM.NAMES` | Y | Y | |
|
||
| `STREAM.LIST` | Y | Y | |
|
||
| `STREAM.PURGE.<name>` | Y | Y | |
|
||
| `STREAM.MSG.GET.<name>` | Y | Y | |
|
||
| `STREAM.MSG.DELETE.<name>` | Y | Y | |
|
||
| `DIRECT.GET.<name>` | Y | Y | Includes direct payload response shape |
|
||
| `CONSUMER.CREATE.<stream>` | Y | Y | |
|
||
| `CONSUMER.INFO.<stream>.<durable>` | Y | Y | |
|
||
| `CONSUMER.DELETE.<stream>.<durable>` | Y | Y | |
|
||
| `CONSUMER.NAMES.<stream>` | Y | Y | |
|
||
| `CONSUMER.LIST.<stream>` | Y | Y | |
|
||
| `CONSUMER.PAUSE.<stream>.<durable>` | Y | Y | |
|
||
| `CONSUMER.RESET.<stream>.<durable>` | Y | Y | |
|
||
| `CONSUMER.UNPIN.<stream>.<durable>` | Y | Y | |
|
||
| `CONSUMER.MSG.NEXT.<stream>.<durable>` | Y | Y | |
|
||
| `STREAM.LEADER.STEPDOWN.<name>` | Y | Y | |
|
||
| `META.LEADER.STEPDOWN` | Y | Y | |
|
||
| `STREAM.SNAPSHOT.<name>` | Y | Y | Snapshot/restore shape implemented; in-memory semantics |
|
||
| `STREAM.RESTORE.<name>` | Y | Y | Snapshot/restore shape implemented; in-memory semantics |
|
||
| `INFO` (account info) | Y | Y | |
|
||
|
||
### Stream Configuration
|
||
|
||
| Option | Go | .NET | Notes |
|
||
|--------|:--:|:----:|-------|
|
||
| Subjects | Y | Y | |
|
||
| Replicas | Y | Y | Wires RAFT replica count |
|
||
| MaxMsgs limit | Y | Y | Enforced via `EnforceLimits()` |
|
||
| Retention (Limits/Interest/WorkQueue) | Y | Y | Runtime dispatch now diverges by contract with work-queue ack-floor enforcement |
|
||
| Discard policy (Old/New) | Y | Y | `Discard=New` now rejects writes when `MaxBytes` is exceeded |
|
||
| MaxBytes / MaxAge (TTL) | Y | Y | Runtime pruning/limits enforced in stream policy paths |
|
||
| MaxMsgsPer (per-subject limit) | Y | Y | Runtime per-subject pruning is enforced |
|
||
| MaxMsgSize | Y | Y | |
|
||
| Storage type selection (Memory/File) | Y | Y | Per-stream backend selection supports memory and file stores |
|
||
| Compression (S2) | Y | Y | |
|
||
| Subject transform | Y | Y | |
|
||
| RePublish | Y | Y | |
|
||
| AllowDirect / KV mode | Y | Y | |
|
||
| Sealed, DenyDelete, DenyPurge | Y | Y | |
|
||
| Duplicates dedup window | Y | Y | Dedup window behavior covered by runtime parity tests |
|
||
|
||
### Consumer Configuration & Delivery
|
||
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Push delivery | Y | Y | `PushConsumerEngine`; basic delivery |
|
||
| Pull fetch | Y | Y | `PullConsumerEngine`; basic batch fetch |
|
||
| Ephemeral consumers | Y | Y | Ephemeral creation auto-generates durable IDs when requested |
|
||
| AckPolicy.None | Y | Y | |
|
||
| AckPolicy.Explicit | Y | Y | `AckProcessor` tracks pending with expiry |
|
||
| AckPolicy.All | Y | Y | Monotonic ack-floor behavior enforced with strict state-machine tests |
|
||
| Redelivery on ack timeout | Y | Y | MaxDeliver floor is enforced (`>=`) with strict redelivery gating |
|
||
| DeliverPolicy (All/Last/New/StartSeq/StartTime) | Y | Y | Runtime policy coverage expanded through strict and long-run parity tests |
|
||
| FilterSubject (single) | Y | Y | |
|
||
| FilterSubjects (multiple) | Y | Y | Multi-filter matching implemented in pull/push delivery paths |
|
||
| MaxAckPending | Y | Y | Pending delivery cap enforced for consumer queues |
|
||
| Idle heartbeat | Y | Y | Push engine emits heartbeat frames for configured consumers |
|
||
| Flow control | Y | Y | |
|
||
| Rate limiting | Y | Y | |
|
||
| Replay policy | Y | Y | Replay timing behavior is validated by runtime parity tests |
|
||
| BackOff (exponential) | Y | Y | |
|
||
|
||
### Storage Backends
|
||
|
||
| Feature | Go FileStore | .NET FileStore | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Append / Load / Purge | Y | Y | Basic JSONL serialization |
|
||
| Recovery on restart | Y | Y | Loads JSONL on startup |
|
||
| Block-based layout (64 MB blocks) | Y | Y | .NET uses flat JSONL; not production-scale |
|
||
| S2 compression | Y | Y | |
|
||
| AES-GCM / ChaCha20 encryption | Y | Y | |
|
||
| Bit-packed sequence indexing | Y | Y | Simple dictionary |
|
||
| TTL / time-based expiry | Y | Y | |
|
||
|
||
MemStore has basic append/load/purge with `Dictionary<long, StoredMessage>` under a lock.
|
||
|
||
### Mirror & Sourcing
|
||
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Mirror consumer creation | Y | Y | `MirrorCoordinator` triggers on append |
|
||
| Mirror sync state tracking | Y | Y | |
|
||
| Source fan-in (multiple sources) | Y | Y | `Sources[]` array support added and replicated via `SourceCoordinator` |
|
||
| Subject mapping for sources | Y | Y | |
|
||
| Cross-account mirror/source | Y | Y | |
|
||
|
||
### RAFT Consensus
|
||
|
||
| Feature | Go (5 037 lines) | .NET (212 lines) | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Leader election / term tracking | Y | Y | In-process; nodes hold direct `List<RaftNode>` references |
|
||
| Log append + quorum | Y | Y | Entries replicated via direct method calls; stale-term append now rejected |
|
||
| Log persistence | Y | Y | Log + term/applied persistence and snapshot-store persistence path validated |
|
||
| Heartbeat / keep-alive | Y | Y | |
|
||
| Log mismatch resolution (NextIndex) | Y | Y | |
|
||
| Snapshot creation | Y | Y | `CreateSnapshotAsync()` exists; stored in-memory |
|
||
| Snapshot network transfer | Y | Y | |
|
||
| Membership changes | Y | Y | |
|
||
| Network RPC transport | Y | Y | `IRaftTransport` path validates quorum-gated commit visibility and vote semantics |
|
||
|
||
### JetStream Clustering
|
||
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Meta-group governance | Y | Y | `JetStreamMetaGroup` tracks streams; no durable consensus |
|
||
| Per-stream replica group | Y | Y | `StreamReplicaGroup` + in-memory RAFT |
|
||
| Asset placement planner | Y | Y | `AssetPlacementPlanner` skeleton |
|
||
| Cross-cluster JetStream (gateways) | Y | Y | Requires functional gateways |
|
||
|
||
---
|
||
|
||
## 12. Clustering
|
||
|
||
> Routes, gateways, and leaf nodes now have account-scoped delivery semantics and idempotent replay coverage in strict-runtime tests.
|
||
|
||
### Routes
|
||
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Listener accept loop | Y | Y | `RouteManager` binds and accepts inbound connections |
|
||
| Outbound seed connections (with backoff) | Y | Y | Iterates `ClusterOptions.Routes` with 250 ms retry |
|
||
| Route handshake (ROUTE `<serverId>`) | Y | Y | Bidirectional: sends own ID, reads peer ID |
|
||
| Remote subscription tracking | Y | Y | `ApplyRemoteSubscription` adds to SubList; `HasRemoteInterest` exposed |
|
||
| Subscription propagation (wire RS+/RS-) | Y | Y | Local SUB/UNSUB is propagated over route wire frames |
|
||
| Message routing (RMSG wire) | Y | Y | Routed publishes forward over RMSG to remote subscribers |
|
||
| RS+/RS- subscription protocol (wire) | Y | Y | Inbound RS+/RS- frames update remote-interest trie |
|
||
| Route pooling (3× per peer) | Y | Y | `ClusterOptions.PoolSize` defaults to 3 links per peer |
|
||
| Account-specific routes | Y | Y | |
|
||
| S2 compression on routes | Y | Y | |
|
||
| CONNECT info + topology gossip | Y | Y | Handshake is two-line text exchange only |
|
||
|
||
### Gateways
|
||
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Any networking (listener / outbound) | Y | Y | Listener + outbound remotes with retry are active |
|
||
| Gateway connection protocol | Y | Y | Baseline `GATEWAY` handshake implemented |
|
||
| Interest-only mode | Y | Y | Baseline A+/A- interest propagation implemented |
|
||
| Reply subject mapping (`_GR_.` prefix) | Y | Y | |
|
||
| Message forwarding to remote clusters | Y | Y | Baseline `GMSG` forwarding implemented |
|
||
|
||
### Leaf Nodes
|
||
|
||
| Feature | Go | .NET | Notes |
|
||
|---------|:--:|:----:|-------|
|
||
| Any networking (listener / spoke) | Y | Y | Listener + outbound remotes with retry are active |
|
||
| Leaf handshake / role negotiation | Y | Y | Baseline `LEAF` handshake implemented |
|
||
| Subscription sharing (LS+/LS-) | Y | Y | LS+/LS- propagation implemented |
|
||
| Loop detection (`$LDS.` prefix) | Y | Y | |
|
||
| Hub-and-spoke account mapping | Y | Y | Baseline LMSG forwarding works; advanced account remapping remains |
|
||
|
||
---
|
||
|
||
## Summary: Remaining Gaps
|
||
|
||
None in the tracked strict full parity scope after this execution cycle.
|
||
|
||
---
|
||
|
||
## 13. JetStream Remaining Parity (2026-02-23)
|
||
|
||
### Newly Ported API Families
|
||
- `$JS.API.INFO`
|
||
- `$JS.API.SERVER.REMOVE`
|
||
- `$JS.API.ACCOUNT.PURGE.*`, `$JS.API.ACCOUNT.STREAM.MOVE.*`, `$JS.API.ACCOUNT.STREAM.MOVE.CANCEL.*`
|
||
- `$JS.API.STREAM.UPDATE.*`, `$JS.API.STREAM.DELETE.*`, `$JS.API.STREAM.NAMES`, `$JS.API.STREAM.LIST`
|
||
- `$JS.API.STREAM.PEER.REMOVE.*`
|
||
- `$JS.API.STREAM.MSG.GET.*`, `$JS.API.STREAM.MSG.DELETE.*`, `$JS.API.STREAM.PURGE.*`
|
||
- `$JS.API.DIRECT.GET.*`
|
||
- `$JS.API.STREAM.SNAPSHOT.*`, `$JS.API.STREAM.RESTORE.*`
|
||
- `$JS.API.CONSUMER.NAMES.*`, `$JS.API.CONSUMER.LIST.*`, `$JS.API.CONSUMER.DELETE.*.*`
|
||
- `$JS.API.CONSUMER.PAUSE.*.*`, `$JS.API.CONSUMER.RESET.*.*`, `$JS.API.CONSUMER.UNPIN.*.*`
|
||
- `$JS.API.CONSUMER.MSG.NEXT.*.*`
|
||
- `$JS.API.CONSUMER.LEADER.STEPDOWN.*.*`
|
||
- `$JS.API.STREAM.LEADER.STEPDOWN.*`, `$JS.API.META.LEADER.STEPDOWN`
|
||
|
||
### Runtime/Storage/RAFT Parity Additions
|
||
- JetStream publish precondition support for expected last sequence (`ErrorCode=10071` on mismatch).
|
||
- Pull consumer `no_wait` contract support (`TimedOut=false` on immediate empty fetch).
|
||
- Ack-all pending floor behavior via `AckProcessor.AckAll` and pending-count introspection.
|
||
- Stream store subject index support (`LoadLastBySubjectAsync`) in `MemStore` and `FileStore`.
|
||
- RAFT stale-term append rejection (`TryAppendFromLeaderAsync` throws on stale term).
|
||
- `/jsz` and `/varz` now expose JetStream API totals/errors from server stats.
|
||
- Route wire protocol path: RS+/RS-/RMSG with default 3-link route pooling.
|
||
- Gateway/Leaf wire protocol paths: A+/A-/GMSG and LS+/LS-/LMSG.
|
||
- Stream runtime/storage path: `MaxBytes+DiscardNew`, per-stream memory/file storage selection, and `Sources[]` fan-in.
|
||
- Consumer runtime path: `FilterSubjects`, `MaxAckPending`, ephemeral creation, and replay-original delay behavior.
|
||
- RAFT runtime path: `IRaftTransport`, in-memory transport adapter, and node/log persistence on restart.
|
||
- Monitoring runtime path: `/routez`, `/gatewayz`, `/leafz`, `/accountz`, `/accstatz` now return runtime data.
|
||
|
||
### Deep Operational Parity Closures (2026-02-23)
|
||
- Truth-matrix guardrails now enforce `differences.md`/parity-map alignment and contradiction detection.
|
||
- Internal JetStream client lifecycle is verified by runtime tests (`JetStreamInternalClientRuntimeTests`).
|
||
- Stream retention/runtime long-run guards now include retention-policy dispatch and dedupe-window expiry coverage.
|
||
- Consumer deliver-policy `LastPerSubject` now resolves the correct subject-scoped cursor.
|
||
- FileStore now persists a block-index manifest and reopens with manifest-backed index recovery.
|
||
- FileStore persisted payloads now use a versioned envelope with key-hash and payload-integrity validation.
|
||
- Deep runtime closure tests now cover flow/replay timing, RAFT append+convergence, governance, and cross-cluster forwarding paths.
|
||
|
||
### Remaining Explicit Deltas
|
||
- None after this deep operational parity cycle; stale contradictory notes were removed.
|
||
|
||
## 14. Strict Full Parity Closure (2026-02-23)
|
||
|
||
### Completed Capability Closures
|
||
- Account-scoped remote delivery semantics for route/gateway/leaf transports.
|
||
- Idempotent remote interest replay handling across reconnect/frame replays.
|
||
- Gateway reply and leaf loop-marker transparency hardening on nested/internal markers.
|
||
- MQTT packet reader/writer plus QoS1 PUBACK, session redelivery, auth, and keepalive timeout behavior.
|
||
- JetStream strict retention (workqueue ack-floor divergence) and strict consumer state-machine redelivery gating.
|
||
- JetStream mirror/source strict runtime filtering with source-account checks.
|
||
- FileStore invariant closure for `LastSeq`/prune/restart consistency.
|
||
- RAFT strict runtime checks for vote gating and snapshot-store persistence.
|
||
- JetStream meta/replica governance strict transition checks.
|
||
- Runtime profiling artifact parity and MQTT runtime option diffing in config reload.
|
||
- Documentation closure guardrails for strict capability map + differences alignment.
|
||
|
||
### Final Verification Evidence
|
||
- `dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~NatsStrictCapabilityInventoryTests|FullyQualifiedName~AccountScopedDeliveryTests|FullyQualifiedName~InterestIdempotencyTests|FullyQualifiedName~RemapRuntimeTests|FullyQualifiedName~LoopTransparencyRuntimeTests|FullyQualifiedName~MqttPacketParserTests|FullyQualifiedName~MqttPacketWriterTests|FullyQualifiedName~MqttSessionRuntimeTests|FullyQualifiedName~MqttQosAckRuntimeTests|FullyQualifiedName~MqttAuthIntegrationTests|FullyQualifiedName~MqttKeepAliveTests|FullyQualifiedName~JetStreamRetentionRuntimeStrictParityTests|FullyQualifiedName~JetStreamConsumerStateMachineStrictParityTests|FullyQualifiedName~JetStreamMirrorSourceStrictRuntimeTests|FullyQualifiedName~JetStreamFileStoreRecoveryStrictParityTests|FullyQualifiedName~JetStreamFileStoreInvariantTests|FullyQualifiedName~RaftStrictConsensusRuntimeTests|FullyQualifiedName~RaftStrictConvergenceRuntimeTests|FullyQualifiedName~JetStreamMetaGovernanceStrictParityTests|FullyQualifiedName~JetStreamReplicaGovernanceStrictParityTests|FullyQualifiedName~PprofRuntimeParityTests|FullyQualifiedName~ConfigRuntimeParityTests|FullyQualifiedName~DifferencesParityClosureTests" -v minimal` → Passed `29`, Failed `0`.
|
||
- `dotnet test -v minimal` → Passed `869`, Failed `0`, Skipped `0`.
|