docs: update differences.md with accurate JetStream and clustering gaps

Add sections 11 (JetStream) and 12 (Clustering) with verified coverage tables. Correct sections 2-4 where ROUTER/GATEWAY/LEAF and RS+/RS-/RMSG were previously marked Y but are partial or stub implementations: - ROUTER: handshake + in-memory subscription tracking only; no RMSG - GATEWAY/LEAF: config-only stubs with no networking - RS+/RS-/RMSG: command matrix only; no wire routing - JetStream: 4 of ~20 API subjects implemented; RAFT is in-memory simulation with no persistence or network transport
2026-02-23 09:56:09 -05:00
parent c7bbf45c8f
commit 9efe787cab
1 changed files with 193 additions and 7 deletions
--- a/differences.md
+++ b/differences.md
@@ -61,9 +61,9 @@
 | Type | Go | .NET | Notes |
 |------|:--:|:----:|-------|
 | CLIENT | Y | Y | |
-| ROUTER | Y | Y | Route handshake + routing primitives implemented |
-| GATEWAY | Y | Y | Gateway manager bootstrap implemented |
-| LEAF | Y | Y | Leaf node manager bootstrap implemented |
+| ROUTER | Y | Partial | Route handshake + in-memory remote subscription tracking; no RMSG message routing, no RS+/RS- wire protocol, no route pooling (3x per peer) |
+| GATEWAY | Y | Stub | Config parsing only; no listener, connections, handshake, interest-only mode, or message forwarding |
+| LEAF | Y | Stub | Config parsing only; no listener, connections, handshake, subscription sharing, or loop detection |
 | SYSTEM (internal) | Y | Y | InternalClient + InternalEventSystem with Channel-based send/receive loops |
 | JETSTREAM (internal) | Y | N | |
 | ACCOUNT (internal) | Y | Y | Lazy per-account InternalClient with import/export subscription support |
@@ -127,9 +127,9 @@ Go implements a sophisticated slow consumer detection system:
 | PING / PONG | Y | Y | |
 | MSG / HMSG | Y | Y | |
 | +OK / -ERR | Y | Y | |
-| RS+/RS-/RMSG (routes) | Y | Y | Route protocol primitives implemented |
+| RS+/RS-/RMSG (routes) | Y | N | Parser/command matrix recognises opcodes; no wire routing — remote subscription propagation uses in-memory method calls; RMSG delivery not implemented |
 | A+/A- (accounts) | Y | N | Inter-server account protocol ops still pending |
-| LS+/LS-/LMSG (leaf) | Y | Y | Leaf protocol primitives implemented |
+| LS+/LS-/LMSG (leaf) | Y | N | Leaf nodes are config-only stubs; no LS+/LS-/LMSG wire protocol handling |

 ### Protocol Parsing Gaps
 | Feature | Go | .NET | Notes |
@@ -191,7 +191,7 @@ Go implements a sophisticated slow consumer detection system:
 |---------|:--:|:----:|-------|
 | Per-account subscription limit | Y | Y | `Account.IncrementSubscriptions()` returns false when `MaxSubscriptions` exceeded |
 | Auto-unsubscribe on max messages | Y | Y | Enforced at delivery; sub removed from trie + client dict when exhausted |
-| Subscription routing propagation | Y | Y | Remote subscription propagation implemented for routes |
+| Subscription routing propagation | Y | Partial | Remote subs tracked in trie; propagation via in-process method calls (no wire RS+/RS-); RMSG forwarding absent |
 | Queue weight (`qw`) field | Y | N | For remote queue load balancing |

 ---
@@ -413,5 +413,191 @@ The following items from the original gap list have been implemented:
 - **System request-reply services** — $SYS.REQ.SERVER.*.VARZ/CONNZ/SUBSZ/HEALTHZ/IDZ/STATSZ with ping wildcards
 - **Account exports/imports** — service and stream imports with ExportAuth, subject transforms, response routing, latency tracking

-### Remaining Lower Priority
+---
+
+## 11. JetStream
+
+> The Go JetStream surface is ~37,500 lines across jetstream.go, stream.go, consumer.go, filestore.go, memstore.go, raft.go. The .NET implementation has ~700 lines covering a minimal subset. Verified by `JetStreamApiRouter.cs` registration.
+
+### JetStream API ($JS.API.* subjects)
+
+| Subject | Go | .NET | Notes |
+|---------|:--:|:----:|-------|
+| `STREAM.CREATE.<name>` | Y | Y | Only API handler implemented |
+| `STREAM.INFO.<name>` | Y | Y | |
+| `STREAM.UPDATE.<name>` | Y | N | |
+| `STREAM.DELETE.<name>` | Y | N | |
+| `STREAM.NAMES` | Y | N | |
+| `STREAM.LIST` | Y | N | |
+| `STREAM.PURGE.<name>` | Y | N | Storage has `PurgeAsync()` but no API handler |
+| `STREAM.MSG.GET.<name>` | Y | N | |
+| `STREAM.MSG.DELETE.<name>` | Y | N | |
+| `DIRECT.GET.<name>` | Y | N | KV-style direct access |
+| `CONSUMER.CREATE.<stream>` | Y | Y | |
+| `CONSUMER.INFO.<stream>.<durable>` | Y | Y | |
+| `CONSUMER.DELETE.<stream>.<durable>` | Y | N | |
+| `CONSUMER.NAMES.<stream>` | Y | N | |
+| `CONSUMER.LIST.<stream>` | Y | N | |
+| `CONSUMER.PAUSE.<stream>.<durable>` | Y | N | |
+| `STREAM.LEADER.STEPDOWN.<name>` | Y | N | |
+| `META.LEADER.STEPDOWN` | Y | N | |
+| `STREAM.SNAPSHOT.<name>` | Y | N | |
+| `STREAM.RESTORE.<name>` | Y | N | |
+| `INFO` (account info) | Y | N | |
+
+### Stream Configuration
+
+| Option | Go | .NET | Notes |
+|--------|:--:|:----:|-------|
+| Subjects | Y | Y | |
+| Replicas | Y | Y | Wires RAFT replica count |
+| MaxMsgs limit | Y | Y | Enforced via `EnforceLimits()` |
+| Retention (Limits/Interest/WorkQueue) | Y | N | Only MaxMsgs trimming; Interest and WorkQueue not implemented |
+| Discard policy (Old/New) | Y | N | |
+| MaxBytes / MaxAge (TTL) | Y | N | |
+| MaxMsgsPer (per-subject limit) | Y | N | |
+| MaxMsgSize | Y | N | |
+| Storage type selection (Memory/File) | Y | N | MemStore default; no config-driven choice |
+| Compression (S2) | Y | N | |
+| Subject transform | Y | N | |
+| RePublish | Y | N | |
+| AllowDirect / KV mode | Y | N | |
+| Sealed, DenyDelete, DenyPurge | Y | N | |
+| Duplicates dedup window | Y | Partial | Dedup ID cache exists; no configurable window |
+
+### Consumer Configuration & Delivery
+
+| Feature | Go | .NET | Notes |
+|---------|:--:|:----:|-------|
+| Push delivery | Y | Partial | `PushConsumerEngine`; basic delivery |
+| Pull fetch | Y | Partial | `PullConsumerEngine`; basic batch fetch |
+| Ephemeral consumers | Y | N | Only durable |
+| AckPolicy.None | Y | Y | |
+| AckPolicy.Explicit | Y | Y | `AckProcessor` tracks pending with expiry |
+| AckPolicy.All | Y | N | |
+| Redelivery on ack timeout | Y | Partial | `NextExpired()` detects expired; limit not enforced |
+| DeliverPolicy (All/Last/New/StartSeq/StartTime) | Y | N | Always delivers from beginning |
+| FilterSubject (single) | Y | Y | |
+| FilterSubjects (multiple) | Y | N | |
+| MaxAckPending | Y | N | |
+| Idle heartbeat | Y | N | Field in model; not implemented |
+| Flow control | Y | N | |
+| Rate limiting | Y | N | |
+| Replay policy | Y | N | |
+| BackOff (exponential) | Y | N | |
+
+### Storage Backends
+
+| Feature | Go FileStore | .NET FileStore | Notes |
+|---------|:--:|:----:|-------|
+| Append / Load / Purge | Y | Y | Basic JSONL serialization |
+| Recovery on restart | Y | Y | Loads JSONL on startup |
+| Block-based layout (64 MB blocks) | Y | N | .NET uses flat JSONL; not production-scale |
+| S2 compression | Y | N | |
+| AES-GCM / ChaCha20 encryption | Y | N | |
+| Bit-packed sequence indexing | Y | N | Simple dictionary |
+| TTL / time-based expiry | Y | N | |
+
+MemStore has basic append/load/purge with `Dictionary<long, StoredMessage>` under a lock.
+
+### Mirror & Sourcing
+
+| Feature | Go | .NET | Notes |
+|---------|:--:|:----:|-------|
+| Mirror consumer creation | Y | Partial | `MirrorCoordinator` triggers on append |
+| Mirror sync state tracking | Y | N | |
+| Source fan-in (multiple sources) | Y | Partial | Single `Source` field; no `Sources[]` array |
+| Subject mapping for sources | Y | N | |
+| Cross-account mirror/source | Y | N | |
+
+### RAFT Consensus
+
+| Feature | Go (5 037 lines) | .NET (212 lines) | Notes |
+|---------|:--:|:----:|-------|
+| Leader election / term tracking | Y | Partial | In-process; nodes hold direct `List<RaftNode>` references |
+| Log append + quorum | Y | Partial | Entries replicated via direct method calls; no network |
+| Log persistence | Y | N | In-memory `List<RaftLogEntry>` only |
+| Heartbeat / keep-alive | Y | N | |
+| Log mismatch resolution (NextIndex) | Y | N | |
+| Snapshot creation | Y | Partial | `CreateSnapshotAsync()` exists; stored in-memory |
+| Snapshot network transfer | Y | N | |
+| Membership changes | Y | N | |
+| Network RPC transport | Y | N | All coordination is in-process |
+
+### JetStream Clustering
+
+| Feature | Go | .NET | Notes |
+|---------|:--:|:----:|-------|
+| Meta-group governance | Y | Partial | `JetStreamMetaGroup` tracks streams; no durable consensus |
+| Per-stream replica group | Y | Partial | `StreamReplicaGroup` + in-memory RAFT |
+| Asset placement planner | Y | Partial | `AssetPlacementPlanner` skeleton |
+| Cross-cluster JetStream (gateways) | Y | N | Requires functional gateways |
+
+---
+
+## 12. Clustering
+
+> Routes, gateways, and leaf nodes are in different states of completeness. All three are present as Go reference feature targets but only routes have any functional networking.
+
+### Routes
+
+| Feature | Go | .NET | Notes |
+|---------|:--:|:----:|-------|
+| Listener accept loop | Y | Y | `RouteManager` binds and accepts inbound connections |
+| Outbound seed connections (with backoff) | Y | Y | Iterates `ClusterOptions.Routes` with 250 ms retry |
+| Route handshake (ROUTE `<serverId>`) | Y | Y | Bidirectional: sends own ID, reads peer ID |
+| Remote subscription tracking | Y | Y | `ApplyRemoteSubscription` adds to SubList; `HasRemoteInterest` exposed |
+| Subscription propagation (in-process) | Y | Partial | `PropagateLocalSubscription` calls peer managers directly — no wire RS+/RS- protocol |
+| Message routing (RMSG wire) | Y | N | **Critical gap**: published messages are never forwarded to remote subscribers |
+| RS+/RS- subscription protocol (wire) | Y | N | Command matrix recognises opcodes but no handler processes inbound RS+/RS- frames |
+| Route pooling (3× per peer) | Y | N | Single connection per remote server ID |
+| Account-specific routes | Y | N | |
+| S2 compression on routes | Y | N | |
+| CONNECT info + topology gossip | Y | N | Handshake is two-line text exchange only |
+
+### Gateways
+
+| Feature | Go | .NET | Notes |
+|---------|:--:|:----:|-------|
+| Any networking (listener / outbound) | Y | N | `GatewayManager.StartAsync()` logs a debug line and zeros a counter |
+| Gateway connection protocol | Y | N | |
+| Interest-only mode | Y | N | |
+| Reply subject mapping (`_GR_.` prefix) | Y | N | |
+| Message forwarding to remote clusters | Y | N | |
+
+### Leaf Nodes
+
+| Feature | Go | .NET | Notes |
+|---------|:--:|:----:|-------|
+| Any networking (listener / spoke) | Y | N | `LeafNodeManager.StartAsync()` logs a debug line and zeros a counter |
+| Leaf handshake / role negotiation | Y | N | |
+| Subscription sharing (LS+/LS-) | Y | N | |
+| Loop detection (`$LDS.` prefix) | Y | N | |
+| Hub-and-spoke account mapping | Y | N | |
+
+---
+
+## Summary: Remaining Gaps
+
+### Clustering (High Impact)
+1. **Route message routing** — Remote subscribers receive no messages; no RMSG implementation
+2. **Gateways** — Non-functional stub; no inter-cluster bridging
+3. **Leaf nodes** — Non-functional stub; no hub/spoke topology
+4. **RS+/RS- wire protocol** — subscription propagation is in-process method calls only
+5. **Route pooling** — single connection per peer vs Go's 3-connection pool
+
+### JetStream (Significant Gaps)
+1. **API coverage** — Only 4 of ~20 `$JS.API.*` subjects handled (STREAM.CREATE/INFO, CONSUMER.CREATE/INFO)
+2. **Stream delete/update/list** — not implemented
+3. **Consumer delete/list/pause** — not implemented
+4. **Retention policies** — only MaxMsgs; Interest and WorkQueue retention absent
+5. **FileStore scalability** — JSONL-based (not block/compressed/encrypted)
+6. **RAFT persistence** — in-memory only; cluster restart loses all JetStream state
+7. **Consumer delivery completeness** — DeliverPolicy, AckPolicy.All, MaxAckPending, heartbeats, flow control absent
+
+### Lower Priority
 1. **Dynamic buffer sizing** — delegated to Pipe, less optimized for long-lived connections
+2. **`plist` optimization** — high-fanout nodes (>256 subs) not converted to array
+3. **External auth callout / proxy auth** — custom auth interfaces not ported
+4. **MQTT listener** — config parsed; no transport
+5. **Inter-server account protocol (A+/A-)** — not implemented