dohertj2/natsdotnet
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: update differences.md with accurate JetStream and clustering gaps

Browse Source 
Add sections 11 (JetStream) and 12 (Clustering) with verified coverage
tables. Correct sections 2-4 where ROUTER/GATEWAY/LEAF and RS+/RS-/RMSG
were previously marked Y but are partial or stub implementations:

- ROUTER: handshake + in-memory subscription tracking only; no RMSG
- GATEWAY/LEAF: config-only stubs with no networking
- RS+/RS-/RMSG: command matrix only; no wire routing
- JetStream: 4 of ~20 API subjects implemented; RAFT is in-memory
  simulation with no persistence or network transport
This commit is contained in:
Joseph Doherty
2026-02-23 09:56:09 -05:00
parent c7bbf45c8f
commit 9efe787cab
1 changed files with 193 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

200
differences.md
View File

@@ -61,9 +61,9 @@
| Type | Go | .NET | Notes | | Type | Go | .NET | Notes |
|------|:--:|:----:|-------| |------|:--:|:----:|-------|
| CLIENT | Y | Y | | | CLIENT | Y | Y | |
| ROUTER | Y | Y | Route handshake + routing primitives implemented | | ROUTER | Y | Partial | Route handshake + in-memory remote subscription tracking; no RMSG message routing, no RS+/RS- wire protocol, no route pooling (3x per peer) |
| GATEWAY | Y | Y | Gateway manager bootstrap implemented | | GATEWAY | Y | Stub | Config parsing only; no listener, connections, handshake, interest-only mode, or message forwarding |
| LEAF | Y | Y | Leaf node manager bootstrap implemented | | LEAF | Y | Stub | Config parsing only; no listener, connections, handshake, subscription sharing, or loop detection |
| SYSTEM (internal) | Y | Y | InternalClient + InternalEventSystem with Channel-based send/receive loops | | SYSTEM (internal) | Y | Y | InternalClient + InternalEventSystem with Channel-based send/receive loops |
| JETSTREAM (internal) | Y | N | | | JETSTREAM (internal) | Y | N | |
| ACCOUNT (internal) | Y | Y | Lazy per-account InternalClient with import/export subscription support | | ACCOUNT (internal) | Y | Y | Lazy per-account InternalClient with import/export subscription support |
@@ -127,9 +127,9 @@ Go implements a sophisticated slow consumer detection system:
| PING / PONG | Y | Y | | | PING / PONG | Y | Y | |
| MSG / HMSG | Y | Y | | | MSG / HMSG | Y | Y | |
| +OK / -ERR | Y | Y | | | +OK / -ERR | Y | Y | |
| RS+/RS-/RMSG (routes) | Y | Y | Route protocol primitives implemented | | RS+/RS-/RMSG (routes) | Y | N | Parser/command matrix recognises opcodes; no wire routing — remote subscription propagation uses in-memory method calls; RMSG delivery not implemented |
| A+/A- (accounts) | Y | N | Inter-server account protocol ops still pending | | A+/A- (accounts) | Y | N | Inter-server account protocol ops still pending |
| LS+/LS-/LMSG (leaf) | Y | Y | Leaf protocol primitives implemented | | LS+/LS-/LMSG (leaf) | Y | N | Leaf nodes are config-only stubs; no LS+/LS-/LMSG wire protocol handling |
### Protocol Parsing Gaps ### Protocol Parsing Gaps
| Feature | Go | .NET | Notes | | Feature | Go | .NET | Notes |
@@ -191,7 +191,7 @@ Go implements a sophisticated slow consumer detection system:
|---------|:--:|:----:|-------| |---------|:--:|:----:|-------|
| Per-account subscription limit | Y | Y | `Account.IncrementSubscriptions()` returns false when `MaxSubscriptions` exceeded | | Per-account subscription limit | Y | Y | `Account.IncrementSubscriptions()` returns false when `MaxSubscriptions` exceeded |
| Auto-unsubscribe on max messages | Y | Y | Enforced at delivery; sub removed from trie + client dict when exhausted | | Auto-unsubscribe on max messages | Y | Y | Enforced at delivery; sub removed from trie + client dict when exhausted |
| Subscription routing propagation | Y | Y | Remote subscription propagation implemented for routes | | Subscription routing propagation | Y | Partial | Remote subs tracked in trie; propagation via in-process method calls (no wire RS+/RS-); RMSG forwarding absent |
| Queue weight (`qw`) field | Y | N | For remote queue load balancing | | Queue weight (`qw`) field | Y | N | For remote queue load balancing |
--- ---
@@ -413,5 +413,191 @@ The following items from the original gap list have been implemented:
- **System request-reply services** — $SYS.REQ.SERVER.*.VARZ/CONNZ/SUBSZ/HEALTHZ/IDZ/STATSZ with ping wildcards - **System request-reply services** — $SYS.REQ.SERVER.*.VARZ/CONNZ/SUBSZ/HEALTHZ/IDZ/STATSZ with ping wildcards
- **Account exports/imports** — service and stream imports with ExportAuth, subject transforms, response routing, latency tracking - **Account exports/imports** — service and stream imports with ExportAuth, subject transforms, response routing, latency tracking
### Remaining Lower Priority ---
## 11. JetStream
> The Go JetStream surface is ~37,500 lines across jetstream.go, stream.go, consumer.go, filestore.go, memstore.go, raft.go. The .NET implementation has ~700 lines covering a minimal subset. Verified by `JetStreamApiRouter.cs` registration.
### JetStream API ($JS.API.* subjects)
| Subject | Go | .NET | Notes |
|---------|:--:|:----:|-------|
| `STREAM.CREATE.<name>` | Y | Y | Only API handler implemented |
| `STREAM.INFO.<name>` | Y | Y | |
| `STREAM.UPDATE.<name>` | Y | N | |
| `STREAM.DELETE.<name>` | Y | N | |
| `STREAM.NAMES` | Y | N | |
| `STREAM.LIST` | Y | N | |
| `STREAM.PURGE.<name>` | Y | N | Storage has `PurgeAsync()` but no API handler |
| `STREAM.MSG.GET.<name>` | Y | N | |
| `STREAM.MSG.DELETE.<name>` | Y | N | |
| `DIRECT.GET.<name>` | Y | N | KV-style direct access |
| `CONSUMER.CREATE.<stream>` | Y | Y | |
| `CONSUMER.INFO.<stream>.<durable>` | Y | Y | |
| `CONSUMER.DELETE.<stream>.<durable>` | Y | N | |
| `CONSUMER.NAMES.<stream>` | Y | N | |
| `CONSUMER.LIST.<stream>` | Y | N | |
| `CONSUMER.PAUSE.<stream>.<durable>` | Y | N | |
| `STREAM.LEADER.STEPDOWN.<name>` | Y | N | |
| `META.LEADER.STEPDOWN` | Y | N | |
| `STREAM.SNAPSHOT.<name>` | Y | N | |
| `STREAM.RESTORE.<name>` | Y | N | |
| `INFO` (account info) | Y | N | |
### Stream Configuration
| Option | Go | .NET | Notes |
|--------|:--:|:----:|-------|
| Subjects | Y | Y | |
| Replicas | Y | Y | Wires RAFT replica count |
| MaxMsgs limit | Y | Y | Enforced via `EnforceLimits()` |
| Retention (Limits/Interest/WorkQueue) | Y | N | Only MaxMsgs trimming; Interest and WorkQueue not implemented |
| Discard policy (Old/New) | Y | N | |
| MaxBytes / MaxAge (TTL) | Y | N | |
| MaxMsgsPer (per-subject limit) | Y | N | |
| MaxMsgSize | Y | N | |
| Storage type selection (Memory/File) | Y | N | MemStore default; no config-driven choice |
| Compression (S2) | Y | N | |
| Subject transform | Y | N | |
| RePublish | Y | N | |
| AllowDirect / KV mode | Y | N | |
| Sealed, DenyDelete, DenyPurge | Y | N | |
| Duplicates dedup window | Y | Partial | Dedup ID cache exists; no configurable window |
### Consumer Configuration & Delivery
| Feature | Go | .NET | Notes |
|---------|:--:|:----:|-------|
| Push delivery | Y | Partial | `PushConsumerEngine`; basic delivery |
| Pull fetch | Y | Partial | `PullConsumerEngine`; basic batch fetch |
| Ephemeral consumers | Y | N | Only durable |
| AckPolicy.None | Y | Y | |
| AckPolicy.Explicit | Y | Y | `AckProcessor` tracks pending with expiry |
| AckPolicy.All | Y | N | |
| Redelivery on ack timeout | Y | Partial | `NextExpired()` detects expired; limit not enforced |
| DeliverPolicy (All/Last/New/StartSeq/StartTime) | Y | N | Always delivers from beginning |
| FilterSubject (single) | Y | Y | |
| FilterSubjects (multiple) | Y | N | |
| MaxAckPending | Y | N | |
| Idle heartbeat | Y | N | Field in model; not implemented |
| Flow control | Y | N | |
| Rate limiting | Y | N | |
| Replay policy | Y | N | |
| BackOff (exponential) | Y | N | |
### Storage Backends
| Feature | Go FileStore | .NET FileStore | Notes |
|---------|:--:|:----:|-------|
| Append / Load / Purge | Y | Y | Basic JSONL serialization |
| Recovery on restart | Y | Y | Loads JSONL on startup |
| Block-based layout (64 MB blocks) | Y | N | .NET uses flat JSONL; not production-scale |
| S2 compression | Y | N | |
| AES-GCM / ChaCha20 encryption | Y | N | |
| Bit-packed sequence indexing | Y | N | Simple dictionary |
| TTL / time-based expiry | Y | N | |
MemStore has basic append/load/purge with `Dictionary<long, StoredMessage>` under a lock.
### Mirror & Sourcing
| Feature | Go | .NET | Notes |
|---------|:--:|:----:|-------|
| Mirror consumer creation | Y | Partial | `MirrorCoordinator` triggers on append |
| Mirror sync state tracking | Y | N | |
| Source fan-in (multiple sources) | Y | Partial | Single `Source` field; no `Sources[]` array |
| Subject mapping for sources | Y | N | |
| Cross-account mirror/source | Y | N | |
### RAFT Consensus
| Feature | Go (5 037 lines) | .NET (212 lines) | Notes |
|---------|:--:|:----:|-------|
| Leader election / term tracking | Y | Partial | In-process; nodes hold direct `List<RaftNode>` references |
| Log append + quorum | Y | Partial | Entries replicated via direct method calls; no network |
| Log persistence | Y | N | In-memory `List<RaftLogEntry>` only |
| Heartbeat / keep-alive | Y | N | |
| Log mismatch resolution (NextIndex) | Y | N | |
| Snapshot creation | Y | Partial | `CreateSnapshotAsync()` exists; stored in-memory |
| Snapshot network transfer | Y | N | |
| Membership changes | Y | N | |
| Network RPC transport | Y | N | All coordination is in-process |
### JetStream Clustering
| Feature | Go | .NET | Notes |
|---------|:--:|:----:|-------|
| Meta-group governance | Y | Partial | `JetStreamMetaGroup` tracks streams; no durable consensus |
| Per-stream replica group | Y | Partial | `StreamReplicaGroup` + in-memory RAFT |
| Asset placement planner | Y | Partial | `AssetPlacementPlanner` skeleton |
| Cross-cluster JetStream (gateways) | Y | N | Requires functional gateways |
---
## 12. Clustering
> Routes, gateways, and leaf nodes are in different states of completeness. All three are present as Go reference feature targets but only routes have any functional networking.
### Routes
| Feature | Go | .NET | Notes |
|---------|:--:|:----:|-------|
| Listener accept loop | Y | Y | `RouteManager` binds and accepts inbound connections |
| Outbound seed connections (with backoff) | Y | Y | Iterates `ClusterOptions.Routes` with 250 ms retry |
| Route handshake (ROUTE `<serverId>`) | Y | Y | Bidirectional: sends own ID, reads peer ID |
| Remote subscription tracking | Y | Y | `ApplyRemoteSubscription` adds to SubList; `HasRemoteInterest` exposed |
| Subscription propagation (in-process) | Y | Partial | `PropagateLocalSubscription` calls peer managers directly — no wire RS+/RS- protocol |
| Message routing (RMSG wire) | Y | N | **Critical gap**: published messages are never forwarded to remote subscribers |
| RS+/RS- subscription protocol (wire) | Y | N | Command matrix recognises opcodes but no handler processes inbound RS+/RS- frames |
| Route pooling (3× per peer) | Y | N | Single connection per remote server ID |
| Account-specific routes | Y | N | |
| S2 compression on routes | Y | N | |
| CONNECT info + topology gossip | Y | N | Handshake is two-line text exchange only |
### Gateways
| Feature | Go | .NET | Notes |
|---------|:--:|:----:|-------|
| Any networking (listener / outbound) | Y | N | `GatewayManager.StartAsync()` logs a debug line and zeros a counter |
| Gateway connection protocol | Y | N | |
| Interest-only mode | Y | N | |
| Reply subject mapping (`_GR_.` prefix) | Y | N | |
| Message forwarding to remote clusters | Y | N | |
### Leaf Nodes
| Feature | Go | .NET | Notes |
|---------|:--:|:----:|-------|
| Any networking (listener / spoke) | Y | N | `LeafNodeManager.StartAsync()` logs a debug line and zeros a counter |
| Leaf handshake / role negotiation | Y | N | |
| Subscription sharing (LS+/LS-) | Y | N | |
| Loop detection (`$LDS.` prefix) | Y | N | |
| Hub-and-spoke account mapping | Y | N | |
---
## Summary: Remaining Gaps
### Clustering (High Impact)
1. **Route message routing** — Remote subscribers receive no messages; no RMSG implementation
2. **Gateways** — Non-functional stub; no inter-cluster bridging
3. **Leaf nodes** — Non-functional stub; no hub/spoke topology
4. **RS+/RS- wire protocol** — subscription propagation is in-process method calls only
5. **Route pooling** — single connection per peer vs Go's 3-connection pool
### JetStream (Significant Gaps)
1. **API coverage** — Only 4 of ~20 `$JS.API.*` subjects handled (STREAM.CREATE/INFO, CONSUMER.CREATE/INFO)
2. **Stream delete/update/list** — not implemented
3. **Consumer delete/list/pause** — not implemented
4. **Retention policies** — only MaxMsgs; Interest and WorkQueue retention absent
5. **FileStore scalability** — JSONL-based (not block/compressed/encrypted)
6. **RAFT persistence** — in-memory only; cluster restart loses all JetStream state
7. **Consumer delivery completeness** — DeliverPolicy, AckPolicy.All, MaxAckPending, heartbeats, flow control absent
### Lower Priority
1. **Dynamic buffer sizing** — delegated to Pipe, less optimized for long-lived connections 1. **Dynamic buffer sizing** — delegated to Pipe, less optimized for long-lived connections
2. **`plist` optimization** — high-fanout nodes (>256 subs) not converted to array
3. **External auth callout / proxy auth** — custom auth interfaces not ported
4. **MQTT listener** — config parsed; no transport
5. **Inter-server account protocol (A+/A-)** — not implemented