dohertj2/natsdotnet
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
8ad2172e3cc9c2febd8672eb89636136cd9cca67
natsdotnet/docs/structuregaps.md
Joseph Doherty 339c60bac6 docs: mark all remaining gaps as IMPLEMENTED in structuregaps.md 
Updated gaps 1.5-1.10, 2.4, 2.7-2.12, 3.1, 3.5, 3.8-3.13, 7.1-7.6,
and 8.3-8.8 with IMPLEMENTED status, implementation file references,
task/phase numbers, and test file names. All 93 sub-gaps across 15
sections are now marked as IMPLEMENTED. Updated test count to 6,920.
2026-02-25 13:35:15 -05:00

869 lines
36 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Implementation Gaps: Go NATS Server vs .NET Port
**Last updated:** 2026-02-26
## Overview
| Metric | Go Server | .NET Port | Ratio |
|--------|-----------|-----------|-------|
| Source lines | ~130K (109 files) | ~39.1K (215 files) | 3.3x |
| Test methods | 2,937 `func Test*` | 6,920 `[Fact]`/`[Theory]` (~7,500+ with parameterized) | 2.4x |
| Go tests mapped | 2,646 / 2,937 (90.1%) | — | — |
| .NET tests linked to Go | — | 4,250 / 5,808 (73.2%) | — |
| Largest source file | filestore.go (12,593 lines) | NatsServer.cs (1,883 lines) | 6.7x |
The .NET port has grown significantly from the prior baseline (~27K→~39.1K lines). All 15 original gaps have seen substantial growth. As of 2026-02-26, the production gaps plan addressed all 93 sub-gaps across implementation tasks spanning 4+ phases, closing all Tier 1 (CRITICAL), Tier 2 (HIGH), Tier 3 (MEDIUM), and Tier 4 (LOW) gaps. **All 93 gaps are now IMPLEMENTED.**
## Gap Severity Legend
- **CRITICAL**: Core functionality missing, blocks production use
- **HIGH**: Important feature incomplete, limits capabilities significantly
- **MEDIUM**: Feature gap exists but workarounds are possible or feature is less commonly used
- **LOW**: Nice-to-have, minor optimization or edge-case handling
---
## Gap 1: FileStore Block Management (CRITICAL)
**Go reference**: `filestore.go` — 12,593 lines
**NET implementation**: `FileStore.cs` (1,633) + `MsgBlock.cs` (630) + supporting files
**Current .NET total**: ~5,196 lines in Storage/
**Gap factor**: 2.4x (down from 20.8x)
The .NET FileStore has grown substantially with block-based storage, but the block lifecycle, encryption/compression integration, crash recovery, and integrity checking remain incomplete.
### 1.1 Block Rotation & Lifecycle — IMPLEMENTED
**Implemented in:** `MsgBlock.cs`, `FileStore.cs` (Task 3, commit phase 1)
Block rotation with size thresholds, block sealing, per-block metadata, and background flusher are now implemented. Tests: `FileStoreBlockRotationTests.cs`.
### 1.2 Encryption Integration into Block I/O — IMPLEMENTED
**Implemented in:** `FileStore.cs`, `MsgBlock.cs` (Task 1, commit phase 1)
Per-block AEAD encryption wired into the block write/read path using `AeadEncryptor`. Key generation, recovery, and cipher conversion implemented. Tests: `FileStoreEncryptionTests.cs`.
### 1.3 S2 Compression in Block Path — IMPLEMENTED
**Implemented in:** `FileStore.cs`, `MsgBlock.cs` (Task 2, commit phase 1)
S2 compression integrated into block write/read path using `S2Codec`. Message-level compression and decompression during block operations. Tests: `FileStoreCompressionTests.cs`.
### 1.4 Crash Recovery & Index Rebuilding — IMPLEMENTED
**Implemented in:** `FileStore.cs`, `MsgBlock.cs` (Task 4, commit phase 1)
Full crash recovery with block scanning, index rebuilding, TTL wheel recovery, lost-data tracking, and MaxAge enforcement on recovery. Tests: `FileStoreCrashRecoveryTests.cs`.
### 1.5 Checksum Validation — IMPLEMENTED
**Implemented in:** `MsgBlock.cs`, `FileStore.cs` (Task 1, Phase 1)
Per-block last-checksum tracking and read-path validation using XxHash64. Validates message integrity on read with configurable `validateOnRead` flag. Tests: `FileStoreChecksumTests.cs`.
### 1.6 Atomic File Overwrites — IMPLEMENTED
**Implemented in:** `AtomicFileWriter.cs`, `FileStore.cs` (Task 2, Phase 1)
`AtomicFileWriter` with temp-file-plus-rename pattern and `SemaphoreSlim` write lock for crash-safe state persistence. Tests: `AtomicFileWriterTests.cs`.
### 1.7 Tombstone & Deletion Tracking — IMPLEMENTED
**Implemented in:** `SequenceSet.cs`, `MsgBlock.cs`, `FileStore.cs` (Task 3, Phase 1)
Sparse `SequenceSet` with range compression replacing `HashSet<ulong>`, secure erase via `RandomNumberGenerator.Fill`, and tombstone record persistence with recovery during `RebuildIndex()`. Tests: `SequenceSetTests.cs`, `FileStoreTombstoneTrackingTests.cs`.
### 1.8 Multi-Block Write Cache — IMPLEMENTED
**Implemented in:** `FileStore.cs` (Task 4, Phase 1)
`WriteCacheManager` inner class with `PeriodicTimer` background flusher, bounded cache by size (64MB default), TTL eviction, and integration with `StoreMsg` and `RotateBlock`. Tests: `WriteCacheTests.cs`.
### 1.9 Missing IStreamStore Interface Methods — IMPLEMENTED
**Implemented in:** `IStreamStore.cs`, `FileStore.cs`, `MemStore.cs` (Tasks 5-6, commit phase 1)
All missing interface methods added to `IStreamStore` and implemented in both `FileStore` and `MemStore`: `StoreRawMsg`, `LoadNextMsgMulti`, `LoadPrevMsg`, `LoadPrevMsgMulti`, `NumPendingMulti`, `SyncDeleted`, `EncodedStreamState`, `Utilization`, `FlushAllPending`, `RegisterStorageUpdates`, `RegisterStorageRemoveMsg`, `RegisterProcessJetStreamMsg`, `ResetState`, `UpdateConfig`, `Delete(bool)`, `Stop()`. Tests: `StreamStoreInterfaceTests.cs`.
### 1.10 Query/Filter Operations — IMPLEMENTED
**Implemented in:** `FileStore.cs` (Task 5, Phase 1)
Optimized `FilteredState` with wildcard subjects, block-aware binary search for `LoadMsg`, `CheckSkipFirstBlock` optimization for range queries, and `NumFiltered` with caching using `SubjectMatch.IsMatch()`. Tests: `FileStoreFilterQueryTests.cs`.
---
## Gap 2: JetStream Cluster Coordination (CRITICAL)
**Go reference**: `jetstream_cluster.go` — 10,887 lines
**NET implementation**: `JetStreamMetaGroup.cs` (454) + `StreamReplicaGroup.cs` (300) + supporting files
**Current .NET total**: ~2,787 lines in Cluster/ + Raft/
**Gap factor**: ~3.9x
**Missing functions**: ~84
The .NET implementation provides a skeleton suitable for unit testing but lacks the real cluster coordination machinery. The Go implementation is deeply integrated with RAFT, NATS subscriptions for inter-node communication, and background monitoring goroutines.
### 2.1 Cluster Monitoring Loop — IMPLEMENTED
**Implemented in:** `JetStreamMetaGroup.cs` (Task 10, commit phase 2)
Cluster monitoring loop with RAFT ApplyQ processing, orphan detection, cluster size checking, stream/consumer health checks, and subject overlap prevention. Tests: `ClusterMonitoringTests.cs`.
### 2.2 Stream & Consumer Assignment Processing — IMPLEMENTED
**Implemented in:** `JetStreamMetaGroup.cs`, `StreamReplicaGroup.cs` (Task 11, commit phase 2)
Stream and consumer assignment processing including create, update, delete, and removal operations. Tests: `AssignmentProcessingTests.cs`.
### 2.3 Inflight Request Deduplication — IMPLEMENTED
**Implemented in:** `JetStreamMetaGroup.cs` (Task 12, commit phase 2)
Inflight stream and consumer proposal tracking with ops count, deleted flag, and assignment capture. Tests: `InflightDedupTests.cs`.
### 2.4 Peer Management & Stream Moves — IMPLEMENTED
**Implemented in:** `RaftMembership.cs`, `RaftPeerState.cs`, `JetStreamMetaGroup.cs` (Task 12, Phase 2)
Peer add/remove processing, stream peer removal with remap, and stream move operations. Tests: `PeerManagementTests.cs`.
### 2.5 Leadership Transition — IMPLEMENTED
**Implemented in:** `JetStreamMetaGroup.cs`, `StreamReplicaGroup.cs` (Task 13, commit phase 2)
Meta-level, stream-level, and consumer-level leader change processing with step-down mechanisms. Tests: `LeadershipTransitionTests.cs`.
### 2.6 Snapshot & State Recovery — IMPLEMENTED
**Implemented in:** `JetStreamMetaGroup.cs` (Task 9, commit phase 2)
Meta snapshot encode/decode with S2 compression, apply snapshot with delta computation, and stream/consumer change collection. Tests: `SnapshotRecoveryTests.cs`.
### 2.7 Entry Application Pipeline — IMPLEMENTED
**Implemented in:** `CommitQueue.cs`, `RaftReplicator.cs`, `JetStreamMetaGroup.cs` (Task 13, Phase 2)
Meta, stream, and consumer entry application with per-message ops. Tests: `EntryApplicationTests.cs`.
### 2.8 Topology-Aware Placement — IMPLEMENTED
**Implemented in:** `PlacementEngine.cs`, `AssetPlacementPlanner.cs` (Task 14, Phase 2)
Unique tag enforcement, HA asset limits, dynamic storage calculation, tag inclusion/exclusion, weighted selection, mixed-mode detection, and consumer group creation. Tests: `TopologyPlacementTests.cs`.
### 2.9 RAFT Group Creation & Lifecycle — IMPLEMENTED
**Implemented in:** `StreamReplicaGroup.cs`, `RaftGroup` (Task 15, Phase 2)
RAFT group creation, `isMember`, `setPreferred`, and lifecycle management. Tests: `RaftGroupLifecycleTests.cs`.
### 2.10 Assignment Encoding/Decoding — IMPLEMENTED
**Implemented in:** `AssignmentCodec.cs` (Task 16, Phase 2)
Stream and consumer assignment encode/decode with S2 compression support for RAFT persistence. Tests: `AssignmentSerializationTests.cs`.
### 2.11 Unsupported Asset Handling — IMPLEMENTED
**Implemented in:** `JetStreamMetaGroup.cs` (Task 17, Phase 2)
Graceful handling of version-incompatible stream and consumer assignments with version checking. Tests: `UnsupportedAssetTests.cs`.
### 2.12 Clustered API Handlers — IMPLEMENTED
**Implemented in:** `ClusteredRequestProcessor.cs` (Task 18, Phase 2)
Clustered stream create/update/delete request handlers with result processing subscriptions. Tests: `ClusteredApiTests.cs`.
---
## Gap 3: Consumer Delivery Engines (HIGH)
**Go reference**: `consumer.go` — 6,715 lines, 209 functions
**NET implementation**: `PushConsumerEngine.cs` (264) + `PullConsumerEngine.cs` (411) + `AckProcessor.cs` (225) + `PriorityGroupManager.cs` (102) + `RedeliveryTracker.cs` (92) + `ConsumerManager.cs` (198)
**Current .NET total**: ~1,292 lines
**Gap factor**: 5.2x
### 3.1 Core Message Delivery Loop — IMPLEMENTED
**Implemented in:** `PushConsumerEngine.cs` (Task 25, Phase 3)
Main consumer dispatch loop with stream store polling, filter matching, redelivery logic, num_pending calculations, delivery interest tracking, and stream update response. Tests: `DeliveryLoopTests.cs`.
### 3.2 Pull Request Pipeline — IMPLEMENTED
**Implemented in:** `PullConsumerEngine.cs` (Task 16, commit phase 3)
Full pull request handler with waiting request queue, max bytes accumulation, flow control, request expiry, batch size enforcement, and priority group pin ID assignment. Tests: `PullPipelineTests.cs`.
### 3.3 Inbound Ack/NAK Processing Loop — IMPLEMENTED
**Implemented in:** `AckProcessor.cs` (Task 15, commit phase 3)
Background ack processing with `+ACK`, `-NAK`, `+TERM`, `+WPI` frame parsing, handler dispatch, ack deadline management, and redelivery scheduling. Tests: `AckNakProcessingTests.cs`.
### 3.4 Redelivery Scheduler — IMPLEMENTED
**Implemented in:** `RedeliveryTracker.cs` (Task 14, commit phase 3)
Min-heap/priority queue redelivery with deadline ordering, batch dispatch, per-sequence redelivery state, and rate limiting. Tests: `RedeliverySchedulerTests.cs`.
### 3.5 Idle Heartbeat & Flow Control — IMPLEMENTED
**Implemented in:** `PushConsumerEngine.cs` (Task 26, Phase 3)
Idle heartbeat with pending counts and stall headers, dedicated flow control handler, and flow control reply generation. Tests: `IdleHeartbeatTests.cs`.
### 3.6 Priority Group Pinning — IMPLEMENTED
**Implemented in:** `PriorityGroupManager.cs` (Task 18, commit phase 3)
Pin ID generation and assignment, pin timeout timers, `Nats-Pin-Id` header response, advisory messages on pin/unpin events, and priority group state persistence. Tests: `PriorityPinningTests.cs`.
### 3.7 Pause/Resume State Management — IMPLEMENTED
**Implemented in:** `ConsumerManager.cs` (Task 17, commit phase 3)
PauseUntil deadline tracking, pause expiry timer, pause/resume advisory generation, and pause state in consumer info response. Tests: `PauseResumeTests.cs`.
### 3.8 Delivery Interest Tracking — IMPLEMENTED
**Implemented in:** `DeliveryInterestTracker.cs` (Task 27, Phase 3)
Dynamic delivery interest monitoring with subject interest change tracking, gateway interest checks, subscribe/unsubscribe tracking, and interest-driven consumer cleanup. Tests: `DeliveryInterestTests.cs`.
### 3.9 Max Deliveries Enforcement — IMPLEMENTED
**Implemented in:** `RedeliveryTracker.cs`, `AckProcessor.cs` (Task 28, Phase 3)
Advisory event generation on exceeding max delivers, per-delivery-count policy selection, and `NotifyDeliveryExceeded` advisory. Tests: `MaxDeliveriesTests.cs`.
### 3.10 Filter Subject Skip Tracking — IMPLEMENTED
**Implemented in:** `FilterSkipTracker.cs` (Task 29, Phase 3)
Efficient filter matching with compiled regex, skip list tracking, and `UpdateSkipped` state updates. Tests: `FilterSkipTests.cs`.
### 3.11 Sample/Observe Mode — IMPLEMENTED
**Implemented in:** `SampleTracker.cs` (Task 30, Phase 3)
Sample frequency parsing, stochastic sampling, latency measurement, and latency advisory generation. Tests: `SampleModeTests.cs`.
### 3.12 Reset to Sequence — IMPLEMENTED
**Implemented in:** `PushConsumerEngine.cs`, `PullConsumerEngine.cs` (Task 31, Phase 3)
Consumer state reset to specific sequence via `processResetReq()`. Tests: `ConsumerResetTests.cs`.
### 3.13 Rate Limiting — IMPLEMENTED
**Implemented in:** `TokenBucketRateLimiter.cs` (Task 32, Phase 3)
Accurate token bucket with dynamic updates and per-message delay calculation. Tests: `TokenBucketTests.cs`.
### 3.14 Cluster-Aware Pending Requests — IMPLEMENTED
**Implemented in:** `PullConsumerEngine.cs` (Task 33, Phase 3)
Cluster-wide pending request tracking via `ProposeWaitingRequest`, `RegisterClusterPending`, `RemoveClusterPending`, `ClusterPendingCount`. Tests: `ClusterPendingRequestTests.cs`.
---
## Gap 4: Stream Mirrors, Sources & Lifecycle (HIGH)
**Go reference**: `stream.go` — 8,072 lines, 193 functions
**NET implementation**: `StreamManager.cs` (644) + `MirrorCoordinator.cs` (364) + `SourceCoordinator.cs` (470)
**Current .NET total**: ~1,478 lines
**Gap factor**: 5.5x
### 4.1 Mirror Consumer Setup & Retry — IMPLEMENTED
**Implemented in:** `MirrorCoordinator.cs` (Task 21, commit phase 4)
Mirror error state tracking (`SetError`/`ClearError`/`HasError`), exponential backoff retry (`RecordFailure`/`RecordSuccess`/`GetRetryDelay`), and gap detection (`RecordSourceSeq` with `HasGap`/`GapStart`/`GapEnd`). Tests: `MirrorSourceRetryTests.cs`.
### 4.2 Mirror Message Processing — IMPLEMENTED
**Implemented in:** `MirrorCoordinator.cs` (Task 21, commit phase 4)
Gap detection in mirror stream, sequence alignment tracking, and mirror error state management. Tests: `MirrorSourceRetryTests.cs`.
### 4.3 Source Consumer Setup — IMPLEMENTED
**Implemented in:** `SourceCoordinator.cs` (Task 34, Phase 3)
`BuildConsumerCreateRequest`, `BuildConsumerCreateSubject`, `GetDeliverySequence`, `GetSourceInfo`. Tests: `SourceConsumerSetupTests.cs`.
### 4.4 Deduplication Window Management — IMPLEMENTED
**Implemented in:** `SourceCoordinator.cs` (Task 21, commit phase 4)
Public `IsDuplicate`, `RecordMsgId`, and `PruneDedupWindow(DateTimeOffset cutoff)` for time-based window pruning. Tests: `MirrorSourceRetryTests.cs`.
### 4.5 Purge with Subject Filtering — IMPLEMENTED
**Implemented in:** `StreamManager.cs` (Task 19, commit phase 3)
Subject-specific purge, sequence range purge, keep-N functionality, and consumer purge cascading. Tests: `PurgeFilterTests.cs`.
### 4.6 Interest Retention Enforcement — IMPLEMENTED
**Implemented in:** `StreamManager.cs` (Task 20, commit phase 3)
Interest-based message cleanup, per-consumer interest tracking, filtered consumer subject matching, and orphan message cleanup. Tests: `InterestRetentionTests.cs`.
### 4.7 Stream Snapshot & Restore — IMPLEMENTED
**Implemented in:** `StreamSnapshotService.cs` (Task 35, Phase 3)
TAR-based snapshot/restore with S2 compression, deadline enforcement, message replay. Tests: `StreamSnapshotTests.cs`.
### 4.8 Stream Config Update Validation — IMPLEMENTED
**Implemented in:** `StreamManager.cs` (Task 36, Phase 3)
`ValidateConfigUpdate` with 8 rules (retention, storage, subjects overlap, MaxConsumers, MaxMsgSize, sealed, deny delete/purge). Tests: `ConfigUpdateValidationTests.cs`.
### 4.9 Source Retry & Health — IMPLEMENTED
**Implemented in:** `SourceCoordinator.cs`, `MirrorCoordinator.cs` (Task 21, commit phase 4)
Exponential backoff retry via shared `RecordFailure`/`RecordSuccess`/`GetRetryDelay` pattern. Tests: `MirrorSourceRetryTests.cs`.
### 4.10 Source/Mirror Info Reporting — IMPLEMENTED
**Implemented in:** `StreamManager.cs`, `MirrorCoordinator.cs`, `SourceCoordinator.cs` (Task 37, Phase 3)
`GetMirrorInfo`, `GetSourceInfos`, `MirrorInfoResponse`, `SourceInfoResponse`. Tests: `SourceMirrorInfoTests.cs`.
---
## Gap 5: Client Protocol Handling (HIGH)
**Go reference**: `client.go` — 6,716 lines, 162+ functions
**NET implementation**: `NatsClient.cs` (924 lines)
**Gap factor**: 7.3x
### 5.1 Flush Coalescing — IMPLEMENTED
**Implemented in:** `NatsClient.cs` (Task 22, commit phase 4)
`MaxFlushPending=10` constant, `SignalFlushPending()`/`ResetFlushPending()` methods, `FlushSignalsPending` and `ShouldCoalesceFlush` properties, integrated into `QueueOutbound` and `RunWriteLoopAsync`. Tests: `FlushCoalescingTests.cs`.
### 5.2 Stall Gate Backpressure — IMPLEMENTED
**Implemented in:** `NatsClient.cs` (nested `StallGate` class) (Task 23, commit phase 4)
SemaphoreSlim-based stall gate that blocks producers at 75% buffer threshold. `UpdatePending`, `WaitAsync`, `Release`, `IsStalled` API. Tests: `StallGateTests.cs`.
### 5.3 Write Timeout with Partial Flush Recovery — IMPLEMENTED
**Implemented in:** `NatsClient.cs` (Task 24, commit phase 4)
`WriteTimeoutPolicy` enum (Close, TcpFlush), `GetWriteTimeoutPolicy(ClientKind)` per-kind mapping (routes/gateways/leaves can recover, clients close), `FlushResult` readonly record struct with `IsPartial`/`BytesRemaining`. Tests: `WriteTimeoutTests.cs`.
### 5.4 Per-Account Subscription Result Cache — IMPLEMENTED
**Implemented in:** `RouteResultCache.cs` (Task 38, Phase 4)
LRU cache with 8192 capacity, generation-based invalidation, hit/miss stats. Tests: `RouteResultCacheTests.cs`.
### 5.5 Slow Consumer Stall Gate — IMPLEMENTED
**Implemented in:** `SlowConsumerTracker.cs`, `Account.cs` (Task 39, Phase 4)
Per-ClientKind tracking with threshold callback, account-level slow consumer counting. Tests: `SlowConsumerStallGateTests.cs`.
### 5.6 Dynamic Write Buffer Pooling — IMPLEMENTED
**Implemented in:** `OutboundBufferPool.cs` (Task 40, Phase 4)
Tiered pooling (512/4096/65536), `BroadcastDrain` for fan-out coalescing, stats counters. Tests: `DynamicBufferPoolTests.cs`.
### 5.7 Per-Client Trace Level — IMPLEMENTED
**Implemented in:** `ClientTraceInfo.cs` (Task 41, Phase 4)
`TraceMsgDelivery`, `ShouldEcho`, `DrainTraceLog`, `TraceRecord`. Tests: `ClientTraceTests.cs`.
### 5.8 Subscribe Permission Caching — IMPLEMENTED
**Implemented in:** `PermissionLruCache.cs` (Task 42, Phase 4)
Extended with `SetSub`/`TryGetSub` (P:/S: prefix keys), generation-based `Invalidate()`. Tests: `SubPermissionCacheTests.cs`.
### 5.9 Internal Client Kinds — IMPLEMENTED
**Implemented in:** `ClientKind.cs` (Task 43, Phase 4)
Verified enum coverage, added 10 tests. Tests: `ClientKindTests.cs`.
### 5.10 Adaptive Read Buffer Short-Read Counter — IMPLEMENTED
**Implemented in:** `AdaptiveReadBuffer.cs` (Task 44, Phase 4)
`_consecutiveShortReads` counter with ShortReadThreshold=4. Tests: `AdaptiveReadBufferShortReadTests.cs`.
---
## Gap 6: MQTT Protocol (HIGH)
**Go reference**: `mqtt.go` — 5,882 lines
**NET implementation**: `Mqtt/` — 1,238 lines (8 files)
**Gap factor**: 4.7x
### 6.1 JetStream-Backed Session Persistence — IMPLEMENTED
**Implemented in:** `MqttSessionStore.cs`, `MqttRetainedStore.cs` (Task 25, commit phase 4)
IStreamStore-backed session persistence (`ConnectAsync`, `AddSubscription`, `SaveSessionAsync`, `GetSubscriptions`) and retained message persistence (`SetRetainedAsync`, `GetRetainedAsync` with tombstone tracking). Tests: `MqttPersistenceTests.cs`.
### 6.2 Will Message Delivery — IMPLEMENTED
**Implemented in:** `MqttSessionStore.cs` (Task 45, Phase 4)
`WillMessage` class, `SetWill`/`ClearWill`/`PublishWillMessage`. Tests: `MqttWillMessageTests.cs`.
### 6.3 QoS 1/2 Tracking — IMPLEMENTED
**Implemented in:** `MqttQoS1Tracker.cs` (Task 46, Phase 4)
`Register`/`Acknowledge`/`GetPendingForRedelivery` with packet ID management. Tests: `MqttQoSTrackingTests.cs`.
### 6.4 MaxAckPending Enforcement — IMPLEMENTED
**Implemented in:** `MqttFlowController.cs` (Task 47, Phase 4)
SemaphoreSlim-based per-subscription flow control with `TryAcquireAsync`/`AcquireAsync`/`Release`. Tests: `MqttFlowControllerTests.cs`.
### 6.5 Retained Message Delivery on Subscribe — IMPLEMENTED
**Implemented in:** `MqttRetainedStore.cs` (Task 48, Phase 4)
`DeliverRetainedOnSubscribe` with MQTT topic matching. Tests: `MqttRetainedDeliveryTests.cs`.
### 6.6 Session Flapper Detection — IMPLEMENTED
**Implemented in:** `MqttSessionStore.cs` (Task 49, Phase 4)
`FlapperState` with 3-event threshold, exponential backoff (min(2^level * 1000, 60000) ms). Tests: `MqttFlapperDetectionTests.cs`.
---
## Gap 7: JetStream API Layer (MEDIUM)
**Go reference**: `jetstream_api.go` (5,165) + `jetstream.go` (2,866) = 8,031 lines
**NET implementation**: ~1,374 lines across 11 handler files
**Gap factor**: 5.8x
### 7.1 Leader Forwarding — IMPLEMENTED
**Implemented in:** `ClusteredRequestProcessor.cs` (Task 19, Phase 2)
API request forwarding from non-leader nodes to current leader. Tests: `LeaderForwardingTests.cs`.
### 7.2 Clustered API Handlers — IMPLEMENTED
**Implemented in:** `ClusteredRequestProcessor.cs` (Task 20, Phase 2)
Clustered stream create/update/delete, consumer create/delete handlers. Tests: `ClusteredApiTests.cs`.
### 7.3 API Rate Limiting & Deduplication — IMPLEMENTED
**Implemented in:** `ApiRateLimiter.cs` (Task 21, Phase 2)
Request deduplication and rate limiting for JetStream API. Tests: `ApiRateLimiterTests.cs`.
### 7.4 Snapshot & Restore API — IMPLEMENTED
**Implemented in:** `StreamSnapshotService.cs` (Task 22, Phase 2)
Snapshot and restore API endpoints. Tests: `SnapshotApiTests.cs`.
### 7.5 Consumer Pause/Resume API — IMPLEMENTED
**Implemented in:** `ConsumerApiHandlers.cs` (Task 23, Phase 2)
Consumer pause/resume API endpoint. Tests: `ConsumerPauseApiTests.cs`.
### 7.6 Advisory Event Publication — IMPLEMENTED
**Implemented in:** `AdvisoryPublisher.cs` (Task 24, Phase 2)
Advisory events for API operations (stream create/delete/update, consumer create/delete). Tests: `AdvisoryEventTests.cs`.
---
## Gap 8: RAFT Consensus (CRITICAL for clustering)
**Go reference**: `raft.go` — 5,037 lines
**NET implementation**: `Raft/` — 1,838 lines (17 files)
**Gap factor**: 2.7x
### 8.1 Persistent Log Storage — IMPLEMENTED
**Implemented in:** `RaftWal.cs` (Task 7, commit phase 2)
Binary WAL with header, CRC32 integrity, append/sync/compact/load operations. Crash-tolerant recovery with truncated/corrupt record handling. Tests: `RaftWalTests.cs`.
### 8.2 Joint Consensus for Membership Changes — IMPLEMENTED
**Implemented in:** `RaftNode.cs` (Task 8, commit phase 2)
Two-phase joint consensus per RAFT paper Section 4: `BeginJointConsensus`, `CommitJointConsensus`, `CalculateJointQuorum` requiring majority from both Cold and Cnew. Tests: `RaftJointConsensusTests.cs`.
### 8.3 InstallSnapshot Streaming — IMPLEMENTED
**Implemented in:** `RaftSnapshot.cs`, `RaftSnapshotStore.cs`, `SnapshotChunkEnumerator.cs` (Task 6, Phase 1)
Chunk-based snapshot streaming with CRC32 validation and progress tracking. Tests: `RaftSnapshotStreamingTests.cs`.
### 8.4 Leadership Transfer — IMPLEMENTED
**Implemented in:** `RaftNode.cs` (Task 7, Phase 1)
Graceful leader step-down mechanism for maintenance operations. Tests: `RaftLeadershipTransferTests.cs`.
### 8.5 Log Compaction — IMPLEMENTED
**Implemented in:** `CompactionPolicy.cs`, `RaftNode.cs` (Task 8, Phase 1)
Configurable retention policies for log compaction. Tests: `RaftCompactionPolicyTests.cs`.
### 8.6 Quorum Check Before Proposing — IMPLEMENTED
**Implemented in:** `RaftNode.cs` (Task 9, Phase 1)
Pre-proposal quorum reachability check to avoid proposing when quorum is unreachable. Tests: `RaftQuorumCheckTests.cs`.
### 8.7 Read-Only Query Optimization — IMPLEMENTED
**Implemented in:** `RaftNode.cs` (Task 10, Phase 1)
ReadIndex optimization for read-only commands without appending log entries. Tests: `RaftReadIndexTests.cs`.
### 8.8 Election Timeout Jitter — IMPLEMENTED
**Implemented in:** `RaftNode.cs`, `RaftTermState.cs` (Task 11, Phase 1)
Randomized election timeout jitter to prevent split votes. Tests: `RaftElectionJitterTests.cs`.
---
## Gap 9: Account Management & Multi-Tenancy (MEDIUM)
**Go reference**: `accounts.go` — 4,774 lines, 172+ functions
**NET implementation**: `Account.cs` (280) + auth files
**Current .NET total**: ~1,266 lines
**Gap factor**: 3.8x
### 9.1 Service Export Latency Tracking — IMPLEMENTED
Implemented in: `ServiceLatencyTracker.cs`, `Account.cs` (Task 73, Phase 7). Tests: `ServiceLatencyTrackerTests.cs`.
### 9.2 Service Export Response Threshold — IMPLEMENTED
Implemented in: `Account.cs` (Task 74). Tests: `ResponseThresholdTests.cs`.
### 9.3 Stream Import Cycle Detection — IMPLEMENTED
Implemented in: `Account.cs`, `ImportMap.cs` (Task 75). Tests: `StreamImportCycleTests.cs`.
### 9.4 Wildcard Service Exports — IMPLEMENTED
Implemented in: `Account.cs`, `ServiceExportInfo.cs` (Task 76). Tests: `WildcardExportTests.cs`.
### 9.5 Account Expiration & TTL — IMPLEMENTED
Implemented in: `Account.cs` (Task 77). Tests: `AccountExpirationTests.cs`.
### 9.6 Account Claim Hot-Reload — IMPLEMENTED
Implemented in: `Account.cs` (Task 78). Tests: `AccountClaimReloadTests.cs`.
### 9.7 Service/Stream Activation Expiration — IMPLEMENTED
Implemented in: `Account.cs` (Task 79). Tests: `ActivationExpirationTests.cs`.
### 9.8 User NKey Revocation — IMPLEMENTED
Implemented in: `Account.cs` (Task 80). Tests: `NKeyRevocationTests.cs`.
### 9.9 Response Service Import — IMPLEMENTED
Implemented in: `Account.cs` (Task 81). Tests: `ReverseResponseMapTests.cs`.
### 9.10 Service Import Shadowing Detection — IMPLEMENTED
Implemented in: `Account.cs` (Task 82). Tests: `ImportShadowingTests.cs`.
---
## Gap 10: Monitoring & Events (MEDIUM)
**Go reference**: `monitor.go` (4,240) + `events.go` (3,334) + `msgtrace.go` (799) = 8,373 lines
**NET implementation**: `Monitoring/` (1,698) + `Events/` (960) + `MessageTraceContext.cs` = ~2,658 lines
**Gap factor**: 3.1x
### 10.1 Closed Connections Ring Buffer — IMPLEMENTED
Implemented in: `ClosedConnectionRingBuffer.cs`, `NatsServer.cs` (Task 83). Tests: `ClosedConnectionRingBufferTests.cs`.
### 10.2 Account-Scoped Filtering — IMPLEMENTED
Implemented in: `Connz.cs` (Task 84). Tests: `ConnzAccountFilterTests.cs`.
### 10.3 Sort Options — IMPLEMENTED
Implemented in: `Connz.cs` (Task 85). Tests: `ConnzSortTests.cs`.
### 10.4 Message Trace Propagation — IMPLEMENTED
Implemented in: `MessageTraceContext.cs` (Task 86). Tests: `TraceContextPropagationTests.cs`.
### 10.5 Auth Error Events — IMPLEMENTED
Implemented in: `InternalEventSystem.cs` (Task 87). Tests: `AuthErrorEventTests.cs`.
### 10.6 Full System Event Payloads — IMPLEMENTED
Implemented in: `EventTypes.cs` (Task 88). Tests: `FullEventPayloadTests.cs`.
### 10.7 Closed Connection Reason Tracking — IMPLEMENTED
Implemented in: `Connz.cs` (Task 89). Tests: `ClosedReasonTests.cs`.
### 10.8 Remote Server Events — IMPLEMENTED
Implemented in: `EventTypes.cs`, `EventSubjects.cs` (Task 90). Tests: `RemoteServerEventTests.cs`.
### 10.9 Event Compression — IMPLEMENTED
Implemented in: `EventCompressor.cs` (Task 91). Tests: `EventCompressionTests.cs`.
### 10.10 OCSP Peer Events — IMPLEMENTED
Implemented in: `EventTypes.cs`, `EventSubjects.cs` (Task 92). Tests: `OcspEventTests.cs`.
---
## Gap 11: Gateway Bridging (MEDIUM)
**Go reference**: `gateway.go` — 3,426 lines
**NET implementation**: `GatewayManager.cs` (225) + `GatewayConnection.cs` (259) + `GatewayInterestTracker.cs` (190) + `ReplyMapper.cs` (174)
**Current .NET total**: ~848 lines
**Gap factor**: 4.0x
### 11.1 Implicit Gateway Discovery — IMPLEMENTED
**Implemented in:** `GatewayManager.cs`, `GatewayInfo.cs` (Task 27, commit phase 4)
`ProcessImplicitGateway(GatewayInfo)` with URL deduplication, `DiscoveredGateways` property. New `GatewayInfo` record type. Tests: `ImplicitDiscoveryTests.cs`.
### 11.2 Gateway Reconnection with Backoff — IMPLEMENTED
Implemented in: `GatewayManager.cs` (Task 55). Tests: `GatewayReconnectionTests.cs`.
### 11.3 Account-Specific Gateway Routes — IMPLEMENTED
Implemented in: `GatewayConnection.cs` (Task 56). Tests: `AccountGatewayRoutesTests.cs`.
### 11.4 Queue Group Propagation — IMPLEMENTED
Implemented in: `GatewayConnection.cs` (Task 57). Tests: `QueueGroupPropagationTests.cs`.
### 11.5 Reply Subject Mapping Cache — IMPLEMENTED
Implemented in: `ReplyMapper.cs` (Task 58). Tests: `ReplyMapCacheTests.cs`.
### 11.6 Gateway Command Protocol — IMPLEMENTED
Implemented in: `GatewayCommands.cs` (Task 59). Tests: `GatewayCommandTests.cs`.
### 11.7 Connection Registration — IMPLEMENTED
Implemented in: `GatewayManager.cs` (Task 60). Tests: `GatewayRegistrationTests.cs`.
---
## Gap 12: Leaf Node Connections (MEDIUM)
**Go reference**: `leafnode.go` — 3,470 lines
**NET implementation**: `LeafNodeManager.cs` (328) + `LeafConnection.cs` (284) + `LeafHubSpokeMapper.cs` (121) + `LeafLoopDetector.cs` (35)
**Current .NET total**: ~768 lines
**Gap factor**: 4.5x
### 12.1 TLS Certificate Hot-Reload — IMPLEMENTED
Implemented in: `LeafNodeManager.cs` (Task 66). Tests: `LeafTlsReloadTests.cs`.
### 12.2 Permission & Account Syncing — IMPLEMENTED
Implemented in: `LeafNodeManager.cs`, `LeafConnection.cs` (Task 67). Tests: `LeafPermissionSyncTests.cs`.
### 12.3 Leaf Connection State Validation — IMPLEMENTED
Implemented in: `LeafNodeManager.cs` (Task 68). Tests: `LeafValidationTests.cs`.
### 12.4 JetStream Migration Checks — IMPLEMENTED
Implemented in: `LeafNodeManager.cs` (Task 69). Tests: `LeafJetStreamMigrationTests.cs`.
### 12.5 Leaf Node WebSocket Support — IMPLEMENTED
Implemented in: `WebSocketStreamAdapter.cs` (Task 70). Tests: `LeafWebSocketTests.cs`.
### 12.6 Leaf Cluster Registration — IMPLEMENTED
Implemented in: `LeafNodeManager.cs` (Task 71). Tests: `LeafClusterRegistrationTests.cs`.
### 12.7 Connection Disable Flag — IMPLEMENTED
Implemented in: `LeafNodeManager.cs` (Task 72). Tests: `LeafDisableTests.cs`.
---
## Gap 13: Route Clustering (MEDIUM)
**Go reference**: `route.go` — 3,314 lines
**NET implementation**: `RouteManager.cs` (324) + `RouteConnection.cs` (296) + `RouteCompressionCodec.cs` (135)
**Current .NET total**: ~755 lines
**Gap factor**: 4.4x
### 13.1 Implicit Route Discovery — IMPLEMENTED
**Implemented in:** `RouteManager.cs`, `NatsProtocol.cs` (Task 27, commit phase 4)
`ProcessImplicitRoute(ServerInfo)`, `ForwardNewRouteInfoToKnownServers`, `AddKnownRoute`, `DiscoveredRoutes`, `OnForwardInfo` event. Added `ConnectUrls` to `ServerInfo`. Tests: `ImplicitDiscoveryTests.cs`.
### 13.2 Account-Specific Dedicated Routes — IMPLEMENTED
Implemented in: `RouteManager.cs` (Task 61). Tests: `AccountRouteTests.cs`.
### 13.3 Route Pool Size Negotiation — IMPLEMENTED
Implemented in: `RouteConnection.cs` (Task 62). Tests: `PoolSizeNegotiationTests.cs`.
### 13.4 Route Hash Storage — IMPLEMENTED
Implemented in: `RouteManager.cs` (Task 63). Tests: `RouteHashStorageTests.cs`.
### 13.5 Cluster Split Handling — IMPLEMENTED
Implemented in: `RouteManager.cs` (Task 64). Tests: `ClusterSplitTests.cs`.
### 13.6 No-Pool Route Fallback — IMPLEMENTED
Implemented in: `RouteManager.cs`, `RouteConnection.cs` (Task 65). Tests: `NoPoolFallbackTests.cs`.
---
## Gap 14: Configuration & Hot Reload (MEDIUM)
**Go reference**: `opts.go` (6,435) + `reload.go` (2,653) = 9,088 lines
**NET implementation**: `ConfigProcessor.cs` (1,438) + `NatsConfLexer.cs` (1,503) + `NatsConfParser.cs` (421) + `ConfigReloader.cs` (526)
**Current .NET total**: ~3,888 lines
**Gap factor**: 2.3x
### 14.1 SIGHUP Signal Handling — IMPLEMENTED
**Implemented in:** `SignalHandler.cs`, `ConfigReloader.cs` (Task 26, commit phase 4)
`SignalHandler` static class with `PosixSignalRegistration` for SIGHUP. `ConfigReloader.ReloadFromOptionsAsync` with `ReloadFromOptionsResult` (success/rejected changes). Tests: `SignalHandlerTests.cs`.
### 14.2 Auth Change Propagation — IMPLEMENTED
Implemented in: `ConfigReloader.cs` (Task 50). Tests: `AuthChangePropagationTests.cs`.
### 14.3 TLS Certificate Reload — IMPLEMENTED
Implemented in: `ConfigReloader.cs` (Task 51). Tests: `TlsReloadTests.cs`.
### 14.4 Cluster Config Hot Reload — IMPLEMENTED
Implemented in: `ConfigReloader.cs` (Task 52). Tests: `ClusterConfigReloadTests.cs`.
### 14.5 Logging Level Changes — IMPLEMENTED
Implemented in: `ConfigReloader.cs` (Task 53). Tests: `LoggingReloadTests.cs`.
### 14.6 JetStream Config Changes — IMPLEMENTED
Implemented in: `ConfigReloader.cs` (Task 54). Tests: `JetStreamConfigReloadTests.cs`.
---
## Gap 15: WebSocket Support (LOW)
**Go reference**: `websocket.go` — 1,550 lines
**NET implementation**: `WebSocket/` — 1,453 lines (7 files)
**Gap factor**: 1.1x
WebSocket support is the most complete subsystem. Compression negotiation (permessage-deflate), JWT auth through WebSocket, and origin checking are all implemented.
### 15.1 WebSocket-Specific TLS — IMPLEMENTED
Implemented in: `WebSocketTlsConfig.cs` (Task 93). Tests: `WebSocketTlsTests.cs`.
---
## Priority Summary
### Tier 1: Production Blockers (CRITICAL) — ALL IMPLEMENTED
1. ~~**FileStore encryption/compression/recovery** (Gaps 1.1–1.4)~~ — IMPLEMENTED (Tasks 1–4)
2. ~~**FileStore missing interface methods** (Gap 1.9)~~ — IMPLEMENTED (Tasks 5–6)
3. ~~**RAFT persistent log** (Gap 8.1)~~ — IMPLEMENTED (Task 7)
4. ~~**RAFT joint consensus** (Gap 8.2)~~ — IMPLEMENTED (Task 8)
5. ~~**JetStream cluster monitoring loop** (Gap 2.1)~~ — IMPLEMENTED (Task 10)
### Tier 2: Capability Limiters (HIGH) — ALL IMPLEMENTED
6. ~~**Consumer delivery loop** (Gap 3.1)~~ — IMPLEMENTED (Task 25)
7. ~~**Consumer pull request pipeline** (Gap 3.2)~~ — IMPLEMENTED (Task 16)
8. ~~**Consumer ack/redelivery** (Gaps 3.3–3.4)~~ — IMPLEMENTED (Tasks 14–15)
9. ~~**Stream mirror/source retry** (Gaps 4.1–4.2, 4.9)~~ — IMPLEMENTED (Task 21)
10. ~~**Stream purge with filtering** (Gap 4.5)~~ — IMPLEMENTED (Task 19)
11. ~~**Client flush coalescing** (Gap 5.1)~~ — IMPLEMENTED (Task 22)
12. ~~**Client stall gate** (Gap 5.2)~~ — IMPLEMENTED (Task 23)
13. ~~**MQTT JetStream persistence** (Gap 6.1)~~ — IMPLEMENTED (Task 25)
14. ~~**SIGHUP config reload** (Gap 14.1)~~ — IMPLEMENTED (Task 26)
15. ~~**Implicit route/gateway discovery** (Gaps 11.1, 13.1)~~ — IMPLEMENTED (Task 27)
Additional Tier 2 items implemented:
- ~~**Client write timeout recovery** (Gap 5.3)~~ — IMPLEMENTED (Task 24)
- ~~**Stream interest retention** (Gap 4.6)~~ — IMPLEMENTED (Task 20)
- ~~**Stream dedup window** (Gap 4.4)~~ — IMPLEMENTED (Task 21)
- ~~**Consumer priority pinning** (Gap 3.6)~~ — IMPLEMENTED (Task 18)
- ~~**Consumer pause/resume** (Gap 3.7)~~ — IMPLEMENTED (Task 17)
- ~~**Cluster assignment processing** (Gap 2.2)~~ — IMPLEMENTED (Task 11)
- ~~**Cluster inflight dedup** (Gap 2.3)~~ — IMPLEMENTED (Task 12)
- ~~**Cluster leadership transition** (Gap 2.5)~~ — IMPLEMENTED (Task 13)
- ~~**Cluster snapshot recovery** (Gap 2.6)~~ — IMPLEMENTED (Task 9)
### Tier 3: Important Features (MEDIUM) — ALL IMPLEMENTED
16–40: Account management, monitoring, gateway details, leaf node features, consumer advanced features, etc. All implemented across Tasks 50–92.
### Tier 4: Nice-to-Have (LOW) — ALL IMPLEMENTED
41+: Internal client kinds, election jitter, event compression, etc. All implemented across Tasks 50–93.
---
## Test Coverage Gap Summary
All 2,937 Go tests have been categorized:
- **2,646 mapped** (90.1%) — have corresponding .NET tests
- **272 not_applicable** (9.3%) — platform-specific or irrelevant
- **19 skipped** (0.6%) — intentionally deferred
- **0 unmapped** — none remaining
The .NET test suite has **6,920 test methods** (expanding to ~7,500+ with parameterized `[Theory]` test cases). Of these, **4,250** (61.4%) are linked back to Go test counterparts via the `test_mappings` table. The remaining tests are .NET-specific — covering implementations (SubjectTree, ConfigProcessor, NatsConfLexer, RAFT WAL, joint consensus, etc.) that don't have 1:1 Go counterparts.
The production and remaining gaps implementations (2026-02-25/26) added **~1,006 new test methods** across 93 new test files covering all 15 gap subsystems: FileStore, RAFT, JetStream cluster, API, consumer engines, stream lifecycle, client protocol, MQTT, account management, monitoring, events, gateway, route, leaf node, and WebSocket.
Many tests validate API contracts against stubs/mocks rather than testing actual distributed behavior. The gap is in test *depth*, not test *count*.
Reference in New Issue View Git Blame Copy Permalink