dohertj2/natsdotnet
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
845441b32c1145e06247c4efe00a57871d13b90e
natsdotnet/gaps/mqtt.md
Joseph Doherty c30e67a69d Fix E2E test gaps and add comprehensive E2E + parity test suites 
- Fix pull consumer fetch: send original stream subject in HMSG (not inbox)
  so NATS client distinguishes data messages from control messages
- Fix MaxAge expiry: add background timer in StreamManager for periodic pruning
- Fix JetStream wire format: Go-compatible anonymous objects with string enums,
  proper offset-based pagination for stream/consumer list APIs
- Add 42 E2E black-box tests (core messaging, auth, TLS, accounts, JetStream)
- Add ~1000 parity tests across all subsystems (gaps closure)
- Update gap inventory docs to reflect implementation status
2026-03-12 14:09:23 -04:00

411 lines
36 KiB
Markdown
Raw Blame History

# MQTT — Gap Analysis
> This file tracks what has and hasn't been ported from Go to .NET for the **MQTT** module.
> See [stillmissing.md](stillmissing.md) for the full LOC comparison across all modules.
## LLM Instructions: How to Analyze This Category
### Step 1: Read the Go Reference Files
Read each Go source file listed below. For every file:
1. Extract all **exported types** (structs, interfaces, type aliases)
2. Extract all **exported methods** on those types (receiver functions)
3. Extract all **exported standalone functions**
4. Note **key constants, enums, and protocol states**
5. Note **important unexported helpers** that implement core logic (functions >20 lines)
6. Pay attention to **concurrency patterns** (goroutines, mutexes, channels) — these map to different .NET patterns
### Step 2: Read the .NET Implementation Files
Read all `.cs` files in the .NET directories listed below. For each Go symbol found in Step 1:
1. Search for a matching type, method, or function in .NET
2. If found, compare the behavior: does it handle the same edge cases? Same error paths?
3. If partially implemented, note what's missing
4. If not found, note it as MISSING
### Step 3: Cross-Reference Tests
Compare Go test functions against .NET test methods:
1. For each Go `Test*` function, check if a corresponding .NET `[Fact]` or `[Theory]` exists
2. Note which test scenarios are covered and which are missing
3. Check the parity DB (`docs/test_parity.db`) for existing mappings:
```bash
sqlite3 docs/test_parity.db "SELECT go_test, dotnet_test, confidence FROM test_mappings tm JOIN go_tests gt ON tm.go_test_id=gt.rowid JOIN dotnet_tests dt ON tm.dotnet_test_id=dt.rowid WHERE gt.go_file LIKE '%PATTERN%'"
```
### Step 4: Classify Each Item
Use these status values:
| Status | Meaning |
|--------|---------|
| **PORTED** | Equivalent exists in .NET with matching behavior |
| **PARTIAL** | .NET implementation exists but is incomplete (missing edge cases, error handling, or features) |
| **MISSING** | No .NET equivalent found — needs to be ported |
| **NOT_APPLICABLE** | Go-specific pattern that doesn't apply to .NET (build tags, platform-specific goroutine tricks, etc.) |
| **DEFERRED** | Intentionally skipped for now (document why) |
### Step 5: Fill In the Gap Inventory
Add rows to the Gap Inventory table below. Group by Go source file. Include the Go file and line number so a porting LLM can jump directly to the reference implementation.
### Key Porting Notes for MQTT
- MQTT runs on top of NATS — MQTT topics map to NATS subjects.
- Session state persists across reconnections (stored in JetStream streams).
- QoS 0 (at most once), QoS 1 (at least once), QoS 2 (exactly once — limited support).
- Retained messages stored as JetStream messages.
- MQTT connections use `ClientKind = MQTT` and have a different parser path.
---
## Go Reference Files (Source)
- `golang/nats-server/server/mqtt.go` — MQTT protocol support (~5,882 lines). MQTT 3.1.1 and 5.0. Session state, QoS levels, retained messages, will messages.
## Go Reference Files (Tests)
- `golang/nats-server/server/mqtt_test.go`
- `golang/nats-server/server/mqtt_ex_test_test.go`
- `golang/nats-server/server/mqtt_ex_bench_test.go`
## .NET Implementation Files (Source)
- `src/NATS.Server/Mqtt/` (all files)
## .NET Implementation Files (Tests)
- `tests/NATS.Server.Tests/Mqtt/`
---
## Gap Inventory
<!-- After analysis, fill in this table. Group rows by Go source file. -->
### golang/nats-server/server/mqtt.go
#### Constants and Protocol Definitions (lines 41-199)
| Go Symbol | Go File:Line | Status | .NET Equivalent | Notes |
|-----------|:-------------|--------|:----------------|-------|
| mqttPacket* constants | mqtt.go:42-57 | PORTED | src/NATS.Server/Mqtt/MqttPacketReader.cs:3-15 | MqttControlPacketType enum covers the packet types |
| mqttProtoLevel | mqtt.go:59 | PORTED | src/NATS.Server/Mqtt/MqttBinaryDecoder.cs:87-88 | Checked in ParseConnect |
| mqttConnFlag* constants | mqtt.go:62-68 | PORTED | src/NATS.Server/Mqtt/MqttBinaryDecoder.cs:96-104 | Decoded inline in ParseConnect |
| mqttPubFlag* constants | mqtt.go:71-75 | PORTED | src/NATS.Server/Mqtt/MqttBinaryDecoder.cs:161-163 | Decoded inline in ParsePublish |
| mqttSubscribeFlags | mqtt.go:78 | PORTED | src/NATS.Server/Mqtt/MqttProtocolConstants.cs:9 | Added SUBSCRIBE flags constant (`0x02`) and wired parser validation in `MqttBinaryDecoder.ParseSubscribe(..., flags)` |
| mqttConnAckRC* constants | mqtt.go:85-91 | PORTED | src/NATS.Server/Mqtt/MqttProtocolConstants.cs:12 | Added MQTT 3.1.1 CONNACK return-code constants (`0x00`..`0x05`) for parity |
| mqttMaxPayloadSize | mqtt.go:94 | PORTED | src/NATS.Server/Mqtt/MqttProtocolConstants.cs:20 | Added max payload constant and wired reader/writer remaining-length guardrails |
| mqttTopicLevelSep, mqttSingleLevelWC, mqttMultiLevelWC | mqtt.go:97-100 | PORTED | src/NATS.Server/Mqtt/MqttBinaryDecoder.cs:254-258 | Used in TranslateFilterToNatsSubject |
| mqttMultiLevelSidSuffix | mqtt.go:105 | PORTED | src/NATS.Server/Mqtt/MqttProtocolConstants.cs:29 | Added `MultiLevelSidSuffix = \" fwc\"` constant |
| mqttPrefix, mqttSubPrefix | mqtt.go:108-113 | PORTED | src/NATS.Server/Mqtt/MqttProtocolConstants.cs:32 | Added MQTT internal subject prefixes (`Prefix`, `SubPrefix`) |
| mqttStreamName, mqttStreamSubjectPrefix | mqtt.go:116-117 | PORTED | src/NATS.Server/Mqtt/MqttProtocolConstants.cs:36 | Added message stream name and subject-prefix constants |
| mqttRetainedMsgsStreamName | mqtt.go:120-121 | PORTED | src/NATS.Server/Mqtt/MqttProtocolConstants.cs:38 | Added retained-message stream constants |
| mqttSessStreamName | mqtt.go:124-125 | PORTED | src/NATS.Server/Mqtt/MqttProtocolConstants.cs:40 | Added session stream constants |
| mqttQoS2IncomingMsgsStreamName | mqtt.go:131-132 | PORTED | src/NATS.Server/Mqtt/MqttProtocolConstants.cs:43 | Added QoS2 incoming stream constants |
| mqttOutStreamName, mqttPubRel* | mqtt.go:135-139 | PORTED | src/NATS.Server/Mqtt/MqttProtocolConstants.cs:47 | Added outbound/PUBREL stream and subject-prefix constants |
| mqttDefaultAckWait | mqtt.go:145 | PORTED | src/NATS.Server/Mqtt/MqttProtocolConstants.cs:23 | Added Go-parity default ack wait (`TimeSpan.FromSeconds(30)`) |
| mqttDefaultMaxAckPending | mqtt.go:149 | PARTIAL | src/NATS.Server/Mqtt/MqttFlowController.cs:15 | Default 1024 matches Go, but not wired to JetStream |
| mqttMaxAckTotalLimit | mqtt.go:153 | PORTED | src/NATS.Server/Mqtt/MqttProtocolConstants.cs:26 | Added max ack total limit constant (`0xFFFF`) for flow/ack accounting parity |
| mqttJSA* token constants | mqtt.go:156-177 | PORTED | src/NATS.Server/Mqtt/MqttProtocolConstants.cs:54 | Added JSA reply prefix, token positions, and stream/consumer/message token constants |
| mqttSessFlappingJailDur | mqtt.go:182 | PARTIAL | src/NATS.Server/Mqtt/MqttSessionStore.cs:96-106 | Flap detection exists but uses different default timing |
| sparkb* constants | mqtt.go:201-211 | PORTED | src/NATS.Server/Mqtt/MqttProtocolConstants.cs:75 | Added Sparkplug birth/death constants and topic-prefix byte arrays |
| mqttNatsHeader* constants | mqtt.go:474-492 | PORTED | src/NATS.Server/Mqtt/MqttProtocolConstants.cs:83 | Added MQTT/NATS re-encoding header-name constants |
#### Core Types (lines 246-498)
| Go Symbol | Go File:Line | Status | .NET Equivalent | Notes |
|-----------|:-------------|--------|:----------------|-------|
| srvMQTT | mqtt.go:246 | PARTIAL | src/NATS.Server/Mqtt/MqttListener.cs:8-12 | Listener exists but no integration with Server struct or authOverride |
| mqttSessionManager | mqtt.go:253-256 | PARTIAL | src/NATS.Server/Mqtt/MqttSessionStore.cs:67 | Session store exists but not multi-account |
| mqttAccountSessionManager | mqtt.go:258-270 | PARTIAL | src/NATS.Server/Mqtt/MqttSessionStore.cs:67 | Single-account only, no JetStream backing, no retained msg sublist, no session hash |
| mqttJSA | mqtt.go:277-289 | PORTED | src/NATS.Server/Mqtt/MqttParityModels.cs (`MqttJsa`) | Added MQTT JetStream API helper model with account/reply-prefix/domain fields |
| mqttJSPubMsg | mqtt.go:291-296 | PORTED | src/NATS.Server/Mqtt/MqttParityModels.cs (`MqttJsPubMsg`) | Added JetStream publish message model (`Subject`, `Payload`, `ReplyTo`) |
| mqttRetMsgDel | mqtt.go:298-301 | PORTED | src/NATS.Server/Mqtt/MqttParityModels.cs (`MqttRetMsgDel`) | Added retained-message delete notification model (`Topic`, `Sequence`) |
| mqttSession | mqtt.go:303-344 | PARTIAL | src/NATS.Server/Mqtt/MqttSessionStore.cs:48-60 | MqttSessionData covers basic fields but missing pendingPublish/pendingPubRel maps, cpending, last_pi, maxp, tmaxack, JetStream consumer tracking |
| mqttPersistedSession | mqtt.go:346-353 | PORTED | src/NATS.Server/Mqtt/MqttParityModels.cs (`MqttPersistedSession`) | Added persisted-session metadata model (`ClientId`, `LastPacketId`, `MaxAckPending`) |
| mqttRetainedMsg | mqtt.go:355-364 | PARTIAL | src/NATS.Server/Mqtt/MqttRetainedStore.cs:14 | Retained message model now includes `Origin`, `Flags`, `Source` in addition to topic/payload. Remaining: cache TTL + JetStream retention parity |
| mqttRetainedMsgRef | mqtt.go:366-369 | PORTED | src/NATS.Server/Mqtt/MqttParityModels.cs (`MqttRetainedMessageRef`) | Added retained-message reference model (`StreamSequence`, `Subject`) |
| mqttSub | mqtt.go:375-391 | PORTED | src/NATS.Server/Mqtt/MqttParityModels.cs (`MqttSub`) | Added MQTT subscription metadata model (`Filter`, `Qos`, `JsDur`, `Prm`, `Reserved`) |
| mqtt (client struct) | mqtt.go:393-408 | PARTIAL | src/NATS.Server/Mqtt/MqttConnection.cs:6-16 | MqttConnection exists but missing reader/writer, asm, sess, cid, rejectQoS2Pub, downgradeQoS2Sub |
| mqttPending | mqtt.go:410-414 | PARTIAL | src/NATS.Server/Mqtt/MqttQoS1Tracker.cs:89-96 | QoS1PendingMessage exists but missing sseq, jsAckSubject, jsDur fields |
| mqttConnectProto | mqtt.go:416-420 | PARTIAL | src/NATS.Server/Mqtt/MqttBinaryDecoder.cs:14-25 | MqttConnectInfo covers most fields but is a record struct, not mutable; missing rd (read deadline) |
| mqttIOReader | mqtt.go:422-425 | NOT_APPLICABLE | | Go-specific IO interface; .NET uses NetworkStream directly |
| mqttReader | mqtt.go:427-433 | PARTIAL | src/NATS.Server/Mqtt/MqttPacketReader.cs:23-41 | MqttPacketReader handles fixed header; missing streaming buffer/partial packet support (pbuf, pstart) |
| mqttWriter | mqtt.go:435-437 | PORTED | src/NATS.Server/Mqtt/MqttPacketWriter.cs:3-38 | MqttPacketWriter covers write operations |
| mqttWill | mqtt.go:439-446 | PARTIAL | src/NATS.Server/Mqtt/MqttSessionStore.cs:35-42 | WillMessage exists but missing subject, mapped byte arrays; topic is string not bytes |
| mqttFilter | mqtt.go:448-453 | PORTED | src/NATS.Server/Mqtt/MqttParityModels.cs (`MqttFilter`) | Added standalone MQTT filter model (`Filter`, `Qos`, `TopicToken`) |
| mqttPublish | mqtt.go:455-463 | PARTIAL | src/NATS.Server/Mqtt/MqttBinaryDecoder.cs:31-37 | MqttPublishInfo covers most fields but missing subject, mapped byte arrays |
| mqttParsedPublishNATSHeader | mqtt.go:495-499 | PORTED | src/NATS.Server/Mqtt/MqttParityModels.cs (`MqttParsedPublishNatsHeader`) | Added parsed NATS-header model for MQTT publish flow (`Subject`, `Mapped`, publish/pubrel flags) |
#### Server Lifecycle Functions (lines 501-722)
| Go Symbol | Go File:Line | Status | .NET Equivalent | Notes |
|-----------|:-------------|--------|:----------------|-------|
| Server.startMQTT | mqtt.go:501 | PARTIAL | src/NATS.Server/Mqtt/MqttListener.cs:27-38 | MqttListener.StartAsync exists but no TLS, no integration with Server lifecycle |
| Server.createMQTTClient | mqtt.go:542 | PARTIAL | src/NATS.Server/Mqtt/MqttListener.cs:130-158 | AcceptLoopAsync creates MqttConnection but no TLS handshake, no auth timer, no read/write loops as goroutines |
| Server.mqttConfigAuth | mqtt.go:666 | MISSING | | Auth configuration override not implemented |
| validateMQTTOptions | mqtt.go:673 | MISSING | | MQTT option validation not implemented |
| client.isMqtt | mqtt.go:726 | NOT_APPLICABLE | | .NET uses MqttConnection as a distinct type rather than a flag on client |
| client.getMQTTClientID | mqtt.go:733 | PARTIAL | src/NATS.Server/Mqtt/MqttConnection.cs:13 | _clientId field exists but no public accessor |
#### Protocol Parsing (lines 742-959)
| Go Symbol | Go File:Line | Status | .NET Equivalent | Notes |
|-----------|:-------------|--------|:----------------|-------|
| client.mqttParse | mqtt.go:742 | PARTIAL | src/NATS.Server/Mqtt/MqttConnection.cs:17-66 | RunAsync handles basic text-based protocol; missing binary MQTT packet dispatch, partial packet handling, read deadline management |
| client.mqttTraceMsg | mqtt.go:961 | MISSING | | MQTT message tracing not implemented |
#### Connection Close and Session Cleanup (lines 976-1107)
| Go Symbol | Go File:Line | Status | .NET Equivalent | Notes |
|-----------|:-------------|--------|:----------------|-------|
| Server.mqttHandleClosedClient | mqtt.go:976 | PARTIAL | src/NATS.Server/Mqtt/MqttConnection.cs:71-77 | DisposeAsync calls Unregister but no session cleanup, no will message handling, no clean session logic |
| Server.mqttUpdateMaxAckPending | mqtt.go:1026 | MISSING | | Max ack pending update across sessions not implemented |
| Server.mqttGetJSAForAccount | mqtt.go:1048 | MISSING | | JetStream API access per account not implemented |
| Server.mqttStoreQoSMsgForAccountOnNewSubject | mqtt.go:1065 | MISSING | | QoS message re-store on subject change not implemented |
| mqttParsePublishNATSHeader | mqtt.go:1080 | MISSING | | NATS header parsing for MQTT messages not implemented |
| mqttParsePubRelNATSHeader | mqtt.go:1096 | MISSING | | PUBREL NATS header parsing not implemented |
#### Account Session Manager (lines 1112-2389)
| Go Symbol | Go File:Line | Status | .NET Equivalent | Notes |
|-----------|:-------------|--------|:----------------|-------|
| Server.getOrCreateMQTTAccountSessionManager | mqtt.go:1112 | MISSING | | Per-account session manager creation not implemented |
| Server.mqttCreateAccountSessionManager | mqtt.go:1152 | MISSING | | JetStream stream/consumer creation for MQTT not implemented (350+ lines) |
| Server.mqttDetermineReplicas | mqtt.go:1502 | MISSING | | Replica count determination for cluster not implemented |
| mqttJSA.newRequest | mqtt.go:1533 | MISSING | | JetStream API request helper not implemented |
| mqttJSA.prefixDomain | mqtt.go:1537 | MISSING | | Domain prefix for JS subjects not implemented |
| mqttJSA.newRequestEx | mqtt.go:1547 | MISSING | | Extended JS API request not implemented |
| mqttJSA.newRequestExMulti | mqtt.go:1566 | MISSING | | Multi-request JS API helper not implemented |
| mqttJSA.sendAck | mqtt.go:1649 | MISSING | | JS ack sending not implemented |
| mqttJSA.sendMsg | mqtt.go:1654 | MISSING | | JS message sending not implemented |
| mqttJSA.createEphemeralConsumer | mqtt.go:1663 | MISSING | | Ephemeral consumer creation not implemented |
| mqttJSA.createDurableConsumer | mqtt.go:1677 | MISSING | | Durable consumer creation not implemented |
| mqttJSA.deleteConsumer | mqtt.go:1692 | MISSING | | Consumer deletion not implemented |
| mqttJSA.createStream | mqtt.go:1707 | MISSING | | Stream creation not implemented |
| mqttJSA.updateStream | mqtt.go:1720 | MISSING | | Stream update not implemented |
| mqttJSA.lookupStream | mqtt.go:1733 | MISSING | | Stream lookup not implemented |
| mqttJSA.deleteStream | mqtt.go:1742 | MISSING | | Stream deletion not implemented |
| mqttJSA.loadLastMsgFor | mqtt.go:1751 | MISSING | | Last message loading not implemented |
| mqttJSA.loadLastMsgForMulti | mqtt.go:1765 | MISSING | | Multi-subject last message loading not implemented |
| mqttJSA.loadNextMsgFor | mqtt.go:1789 | MISSING | | Next message loading not implemented |
| mqttJSA.loadMsg | mqtt.go:1803 | MISSING | | Message loading by seq not implemented |
| mqttJSA.storeMsgNoWait | mqtt.go:1817 | MISSING | | No-wait message store not implemented |
| mqttJSA.storeMsg | mqtt.go:1825 | MISSING | | Message store not implemented |
| mqttJSA.storeSessionMsg | mqtt.go:1834 | MISSING | | Session message store not implemented |
| mqttJSA.loadSessionMsg | mqtt.go:1848 | MISSING | | Session message load not implemented |
| mqttJSA.deleteMsg | mqtt.go:1853 | MISSING | | Message deletion not implemented |
| isErrorOtherThan | mqtt.go:1879 | NOT_APPLICABLE | | Go error-checking helper; .NET uses exception types |
| mqttAccountSessionManager.processJSAPIReplies | mqtt.go:1886 | MISSING | | JS API reply processing callback not implemented |
| mqttAccountSessionManager.processRetainedMsg | mqtt.go:1971 | MISSING | | Retained message processing callback not implemented |
| mqttAccountSessionManager.processRetainedMsgDel | mqtt.go:2006 | MISSING | | Retained message delete callback not implemented |
| mqttAccountSessionManager.processSessionPersist | mqtt.go:2030 | MISSING | | Session persist callback not implemented |
| mqttAccountSessionManager.addSessToFlappers | mqtt.go:2092 | PARTIAL | src/NATS.Server/Mqtt/MqttSessionStore.cs:251 | TrackConnectDisconnect exists but uses different data structures |
| mqttAccountSessionManager.removeSessFromFlappers | mqtt.go:2116 | PARTIAL | src/NATS.Server/Mqtt/MqttSessionStore.cs:322 | ClearFlapperState exists |
| mqttAccountSessionManager.createSubscription | mqtt.go:2124 | MISSING | | Internal subscription creation not implemented |
| mqttAccountSessionManager.cleanupRetainedMessageCache | mqtt.go:2140 | MISSING | | Retained message cache cleanup timer not implemented |
| mqttAccountSessionManager.sendJSAPIrequests | mqtt.go:2174 | MISSING | | JS API request sending goroutine not implemented |
| mqttAccountSessionManager.addRetainedMsg | mqtt.go:2269 | PARTIAL | src/NATS.Server/Mqtt/MqttRetainedStore.cs:51-62 | SetRetained exists but no sseq tracking, no sublist insertion |
| mqttAccountSessionManager.removeRetainedMsg | mqtt.go:2304 | PARTIAL | src/NATS.Server/Mqtt/MqttRetainedStore.cs:53-55 | SetRetained with empty payload removes, but no seq/sublist management |
| mqttAccountSessionManager.lockSession | mqtt.go:2326 | MISSING | | Session locking mechanism not implemented |
| mqttAccountSessionManager.unlockSession | mqtt.go:2345 | MISSING | | Session unlocking mechanism not implemented |
| mqttAccountSessionManager.addSession | mqtt.go:2356 | PARTIAL | src/NATS.Server/Mqtt/MqttSessionStore.cs:192-196 | SaveSession exists but no hash-based lookup |
| mqttAccountSessionManager.removeSession | mqtt.go:2372 | PARTIAL | src/NATS.Server/Mqtt/MqttSessionStore.cs:209-210 | DeleteSession exists but no hash-based removal |
#### Subscription Processing (lines 2389-2730)
| Go Symbol | Go File:Line | Status | .NET Equivalent | Notes |
|-----------|:-------------|--------|:----------------|-------|
| mqttSession.processQOS12Sub | mqtt.go:2389 | MISSING | | QoS 1/2 subscription processing with JS consumer not implemented |
| mqttSession.processSub | mqtt.go:2396 | PARTIAL | src/NATS.Server/Mqtt/MqttListener.cs:40-43 | RegisterSubscription exists but extremely simplified; no QoS handling, no JS consumer, no retained message delivery |
| mqttAccountSessionManager.processSubs | mqtt.go:2463 | MISSING | | Bulk subscription processing with JS consumers not implemented (180 lines) |
| mqttAccountSessionManager.serializeRetainedMsgsForSub | mqtt.go:2642 | MISSING | | Retained message serialization for new subscriptions not implemented |
| mqttAccountSessionManager.addRetainedSubjectsForSubject | mqtt.go:2700 | MISSING | | Retained subject matching via sublist not implemented |
| mqttAccountSessionManager.loadRetainedMessages | mqtt.go:2730 | MISSING | | Retained message loading from JetStream not implemented |
#### Retained Message Encoding/Decoding (lines 2798-2960)
| Go Symbol | Go File:Line | Status | .NET Equivalent | Notes |
|-----------|:-------------|--------|:----------------|-------|
| mqttEncodeRetainedMessage | mqtt.go:2798 | MISSING | | Retained message NATS encoding with headers not implemented |
| mqttSliceHeaders | mqtt.go:2855 | MISSING | | NATS header slicing utility not implemented |
| mqttDecodeRetainedMessage | mqtt.go:2908 | MISSING | | Retained message NATS decoding not implemented |
#### Session CRUD (lines 2972-3190)
| Go Symbol | Go File:Line | Status | .NET Equivalent | Notes |
|-----------|:-------------|--------|:----------------|-------|
| mqttAccountSessionManager.createOrRestoreSession | mqtt.go:2972 | PARTIAL | src/NATS.Server/Mqtt/MqttSessionStore.cs:370-397 | ConnectAsync handles basic create/restore but no JetStream persistence, no hash-based lookup |
| mqttAccountSessionManager.deleteRetainedMsg | mqtt.go:3011 | MISSING | | JetStream retained message deletion not implemented |
| mqttAccountSessionManager.notifyRetainedMsgDeleted | mqtt.go:3018 | MISSING | | Retained message delete notification not implemented |
| mqttAccountSessionManager.transferUniqueSessStreamsToMuxed | mqtt.go:3031 | MISSING | | Session stream migration not implemented |
| mqttAccountSessionManager.transferRetainedToPerKeySubjectStream | mqtt.go:3089 | MISSING | | Retained message stream migration not implemented |
| mqttAccountSessionManager.getCachedRetainedMsg | mqtt.go:3146 | MISSING | | Retained message cache get not implemented |
| mqttAccountSessionManager.setCachedRetainedMsg | mqtt.go:3169 | MISSING | | Retained message cache set not implemented |
| mqttSessionCreate | mqtt.go:3193 | PARTIAL | src/NATS.Server/Mqtt/MqttSessionStore.cs:48-60 | MqttSessionData creation exists but missing maxp, pubRelSubject, idHash |
| mqttSession.save | mqtt.go:3215 | PARTIAL | src/NATS.Server/Mqtt/MqttSessionStore.cs:417-425 | SaveSessionAsync exists but no expected-last-sequence header, simplified JSON |
| mqttSession.clear | mqtt.go:3259 | PARTIAL | src/NATS.Server/Mqtt/MqttSessionStore.cs:209-210 | DeleteSession exists but no JetStream consumer cleanup |
| mqttSession.update | mqtt.go:3316 | PARTIAL | src/NATS.Server/Mqtt/MqttSessionStore.cs:402-411 | AddSubscription exists but no filter-level granularity, no auto-persist |
| mqttSession.bumpPI | mqtt.go:3345 | PARTIAL | src/NATS.Server/Mqtt/MqttQoS1Tracker.cs:75-83 | GetNextPacketId exists but simpler (no collision check with pendingPubRel) |
| mqttSession.trackPublishRetained | mqtt.go:3375 | MISSING | | Retained message publish tracking not implemented |
| mqttSession.trackPublish | mqtt.go:3400 | PARTIAL | src/NATS.Server/Mqtt/MqttQoS1Tracker.cs:27-39 | Register exists but no JS ack subject tracking, no cpending map |
| mqttSession.untrackPublish | mqtt.go:3478 | PARTIAL | src/NATS.Server/Mqtt/MqttQoS1Tracker.cs:45-48 | Acknowledge exists but does not return JS ack subject |
| mqttSession.trackAsPubRel | mqtt.go:3503 | MISSING | | PUBREL tracking not implemented |
| mqttSession.untrackPubRel | mqtt.go:3542 | MISSING | | PUBREL untracking not implemented |
| mqttSession.deleteConsumer | mqtt.go:3562 | MISSING | | JS consumer deletion from session not implemented |
#### CONNECT Processing (lines 3576-3988)
| Go Symbol | Go File:Line | Status | .NET Equivalent | Notes |
|-----------|:-------------|--------|:----------------|-------|
| client.mqttParseConnect | mqtt.go:3576 | PARTIAL | src/NATS.Server/Mqtt/MqttBinaryDecoder.cs:70-140 | ParseConnect handles binary parsing well, but missing protocol level validation (rejects non-4), QoS2 will rejection, UTF-8 validation, subject mapping |
| client.mqttConnectTrace | mqtt.go:3753 | MISSING | | Connect trace formatting not implemented |
| Server.mqttProcessConnect | mqtt.go:3792 | PARTIAL | src/NATS.Server/Mqtt/MqttConnection.cs:34-48 | Connect handling exists but extremely simplified; no account session manager, no session restore, no flapper detection, no existing client eviction, no JetStream integration |
| client.mqttEnqueueConnAck | mqtt.go:3976 | PARTIAL | src/NATS.Server/Mqtt/MqttConnection.cs:45 | Writes "CONNACK" text; missing binary CONNACK with session-present flag and return code |
| Server.mqttHandleWill | mqtt.go:3990 | PARTIAL | src/NATS.Server/Mqtt/MqttSessionStore.cs:164-178 | PublishWillMessage exists but not integrated into connection close flow |
#### PUBLISH Processing (lines 4023-4560)
| Go Symbol | Go File:Line | Status | .NET Equivalent | Notes |
|-----------|:-------------|--------|:----------------|-------|
| client.mqttParsePub | mqtt.go:4023 | PARTIAL | src/NATS.Server/Mqtt/MqttBinaryDecoder.cs:155-185 | ParsePublish handles basic parsing; missing topic-to-subject conversion, subject mapping, empty topic validation |
| mqttPubTrace | mqtt.go:4096 | MISSING | | Publish trace formatting not implemented |
| mqttComputeNatsMsgSize | mqtt.go:4113 | MISSING | | NATS message size computation not implemented |
| mqttNewDeliverableMessage | mqtt.go:4139 | MISSING | | NATS deliverable message composition not implemented |
| mqttNewDeliverablePubRel | mqtt.go:4183 | MISSING | | PUBREL deliverable message composition not implemented |
| Server.mqttProcessPub | mqtt.go:4201 | PARTIAL | src/NATS.Server/Mqtt/MqttConnection.cs:53-63 | Basic QoS0/QoS1 publish exists; missing max payload check, QoS2 flow, JetStream message storage |
| Server.mqttInitiateMsgDelivery | mqtt.go:4249 | PARTIAL | src/NATS.Server/Mqtt/MqttListener.cs:46-58 | PublishAsync delivers to subscribers; missing NATS header encoding, permission check, JetStream storage for QoS1+ |
| Server.mqttStoreQoS2MsgOnce | mqtt.go:4295 | MISSING | | QoS2 message deduplication and store not implemented |
| client.mqttQoS2InternalSubject | mqtt.go:4318 | MISSING | | QoS2 internal subject composition not implemented |
| Server.mqttProcessPubRel | mqtt.go:4326 | PARTIAL | src/NATS.Server/Mqtt/MqttRetainedStore.cs:264 | ProcessPubRel exists in QoS2StateMachine but not wired to JetStream or message delivery |
| client.mqttHandlePubRetain | mqtt.go:4366 | PARTIAL | src/NATS.Server/Mqtt/MqttRetainedStore.cs:51-62 | SetRetained handles basic store/clear but no Sparkplug B, no JetStream persistence, no network notification |
| Server.mqttCheckPubRetainedPerms | mqtt.go:4471 | MISSING | | Retained message permission checking not implemented |
| generatePubPerms | mqtt.go:4564 | MISSING | | Publish permission generation not implemented |
| pubAllowed | mqtt.go:4595 | MISSING | | Publish permission checking not implemented |
| client.mqttEnqueuePubResponse | mqtt.go:4609 | PARTIAL | src/NATS.Server/Mqtt/MqttConnection.cs:58 | Writes "PUBACK" text for QoS1; missing binary PUBACK/PUBREC/PUBREL/PUBCOMP encoding |
| mqttParsePIPacket | mqtt.go:4641 | PARTIAL | src/NATS.Server/Mqtt/MqttBinaryDecoder.cs:316-323 | ReadUInt16BigEndian reads packet ID but no zero-check wrapper |
| client.mqttProcessPublishReceived | mqtt.go:4656 | MISSING | | PUBACK/PUBREC processing with JS ack not implemented |
| client.mqttProcessPubAck | mqtt.go:4691 | PARTIAL | src/NATS.Server/Mqtt/MqttConnection.cs:62-63 | AckPendingPublish exists but no JS ack, no PUBREL tracking |
| client.mqttProcessPubRec | mqtt.go:4695 | PARTIAL | src/NATS.Server/Mqtt/MqttRetainedStore.cs:248 | ProcessPubRec exists in QoS2StateMachine but not wired to session or JetStream |
| client.mqttProcessPubComp | mqtt.go:4700 | PARTIAL | src/NATS.Server/Mqtt/MqttRetainedStore.cs:281 | ProcessPubComp exists in QoS2StateMachine but not wired to session or JetStream |
| mqttGetQoS | mqtt.go:4720 | PORTED | src/NATS.Server/Mqtt/MqttBinaryDecoder.cs:162 | Inline in ParsePublish |
| mqttIsRetained | mqtt.go:4724 | PORTED | src/NATS.Server/Mqtt/MqttBinaryDecoder.cs:163 | Inline in ParsePublish |
| sparkbParseBirthDeathTopic | mqtt.go:4728 | MISSING | | Sparkplug B birth/death topic parsing not implemented |
#### SUBSCRIBE/UNSUBSCRIBE Processing (lines 4760-5557)
| Go Symbol | Go File:Line | Status | .NET Equivalent | Notes |
|-----------|:-------------|--------|:----------------|-------|
| client.mqttParseSubs | mqtt.go:4760 | PARTIAL | src/NATS.Server/Mqtt/MqttBinaryDecoder.cs:197-217 | ParseSubscribe exists but no flag validation, no UTF-8 validation |
| client.mqttParseSubsOrUnsubs | mqtt.go:4764 | PARTIAL | src/NATS.Server/Mqtt/MqttBinaryDecoder.cs:197-217 | Combined parse exists only for subscribe; no unsubscribe binary parser |
| mqttSubscribeTrace | mqtt.go:4826 | MISSING | | Subscribe trace formatting not implemented |
| mqttDeliverMsgCbQoS0 | mqtt.go:4863 | MISSING | | QoS0 delivery callback not implemented (71 lines; core delivery logic) |
| mqttDeliverMsgCbQoS12 | mqtt.go:4934 | MISSING | | QoS1/2 delivery callback not implemented (70+ lines; core delivery logic) |
| mqttDeliverPubRelCb | mqtt.go:5004 | MISSING | | PUBREL delivery callback not implemented |
| mqttMustIgnoreForReservedSub | mqtt.go:5038 | MISSING | | Reserved subject ignore check not implemented |
| isMQTTReservedSubscription | mqtt.go:5047 | MISSING | | Reserved subscription check not implemented |
| sparkbReplaceDeathTimestamp | mqtt.go:5058 | MISSING | | Sparkplug B death timestamp replacement not implemented |
| client.mqttEnqueuePublishMsgTo | mqtt.go:5107 | MISSING | | MQTT PUBLISH message serialization and enqueue not implemented |
| mqttWriter.WritePublishHeader | mqtt.go:5156 | MISSING | | PUBLISH header serialization not implemented (MqttPacketWriter has generic Write but not PUBLISH-specific) |
| mqttMakePublishHeader | mqtt.go:5184 | MISSING | | PUBLISH header creation not implemented |
| client.mqttProcessSubs | mqtt.go:5204 | PARTIAL | src/NATS.Server/Mqtt/MqttListener.cs:40-43 | RegisterSubscription exists but no session locking, no JS consumer, no QoS handling |
| mqttSession.cleanupFailedSub | mqtt.go:5224 | MISSING | | Failed subscription cleanup not implemented |
| mqttSession.ensurePubRelConsumerSubscription | mqtt.go:5238 | MISSING | | PUBREL consumer subscription setup not implemented (90 lines) |
| mqttSession.processJSConsumer | mqtt.go:5327 | MISSING | | JS consumer creation/restoration for subscription not implemented (100+ lines) |
| client.mqttSendRetainedMsgsToNewSubs | mqtt.go:5435 | PARTIAL | src/NATS.Server/Mqtt/MqttRetainedStore.cs:98-104 | DeliverRetainedOnSubscribe exists but not integrated into connection flow |
| client.mqttEnqueueSubAck | mqtt.go:5449 | PARTIAL | src/NATS.Server/Mqtt/MqttConnection.cs:51 | Writes "SUBACK" text; missing binary SUBACK with per-filter QoS return codes |
| client.mqttParseUnsubs | mqtt.go:5469 | MISSING | | Binary UNSUBSCRIBE packet parsing not implemented |
| client.mqttProcessUnsubs | mqtt.go:5480 | MISSING | | UNSUBSCRIBE processing with JS consumer cleanup not implemented |
| client.mqttEnqueueUnsubAck | mqtt.go:5531 | MISSING | | Binary UNSUBACK packet encoding not implemented |
| mqttUnsubscribeTrace | mqtt.go:5541 | MISSING | | Unsubscribe trace formatting not implemented |
#### PING/PONG (lines 5565-5576)
| Go Symbol | Go File:Line | Status | .NET Equivalent | Notes |
|-----------|:-------------|--------|:----------------|-------|
| client.mqttEnqueuePingResp | mqtt.go:5565 | MISSING | | Binary PINGRESP encoding not implemented (text protocol only has no PING support) |
#### Trace and Error Helpers (lines 5577-5582)
| Go Symbol | Go File:Line | Status | .NET Equivalent | Notes |
|-----------|:-------------|--------|:----------------|-------|
| errOrTrace | mqtt.go:5577 | NOT_APPLICABLE | | Go-specific trace helper; .NET uses ILogger |
#### Subject/Topic Conversion (lines 5592-5739)
| Go Symbol | Go File:Line | Status | .NET Equivalent | Notes |
|-----------|:-------------|--------|:----------------|-------|
| mqttTopicToNATSPubSubject | mqtt.go:5592 | PARTIAL | src/NATS.Server/Mqtt/MqttBinaryDecoder.cs:246-264 | TranslateFilterToNatsSubject exists but uses simple char-by-char replace; missing dot-to-double-slash, empty level handling, space rejection, leading/trailing slash handling |
| mqttFilterToNATSSubject | mqtt.go:5598 | PARTIAL | src/NATS.Server/Mqtt/MqttBinaryDecoder.cs:246-264 | Same as above; wildcard-ok flag not separated |
| mqttToNATSSubjectConversion | mqtt.go:5619 | PARTIAL | src/NATS.Server/Mqtt/MqttBinaryDecoder.cs:246-264 | Core conversion exists but missing edge cases: '.' to '//', leading '/' to '/.', trailing '/' to './', empty levels to './', space error |
| natsSubjectStrToMQTTTopic | mqtt.go:5694 | MISSING | | NATS subject to MQTT topic (string variant) not implemented |
| natsSubjectToMQTTTopic | mqtt.go:5698 | MISSING | | NATS subject to MQTT topic (byte variant) not implemented |
| mqttNeedSubForLevelUp | mqtt.go:5730 | MISSING | | Level-up subscription check for '#' wildcard not implemented |
#### Reader Functions (lines 5747-5842)
| Go Symbol | Go File:Line | Status | .NET Equivalent | Notes |
|-----------|:-------------|--------|:----------------|-------|
| mqttReader.reset | mqtt.go:5747 | MISSING | | Streaming reader reset with partial buffer not implemented |
| mqttReader.hasMore | mqtt.go:5760 | NOT_APPLICABLE | | .NET uses Span-based parsing, no position tracking needed |
| mqttReader.readByte | mqtt.go:5764 | PORTED | src/NATS.Server/Mqtt/MqttPacketReader.cs:30-31 | Buffer indexing in Read method |
| mqttReader.readPacketLen / readPacketLenWithCheck | mqtt.go:5773-5803 | PORTED | src/NATS.Server/Mqtt/MqttPacketReader.cs:43-62 | DecodeRemainingLength implemented |
| mqttReader.readString | mqtt.go:5805 | PORTED | src/NATS.Server/Mqtt/MqttBinaryDecoder.cs:275-289 | ReadUtf8String implemented |
| mqttReader.readBytes | mqtt.go:5814 | PORTED | src/NATS.Server/Mqtt/MqttBinaryDecoder.cs:296-310 | ReadBinaryField implemented |
| mqttReader.readUint16 | mqtt.go:5835 | PORTED | src/NATS.Server/Mqtt/MqttBinaryDecoder.cs:316-323 | ReadUInt16BigEndian implemented |
#### Writer Functions (lines 5850-5882)
| Go Symbol | Go File:Line | Status | .NET Equivalent | Notes |
|-----------|:-------------|--------|:----------------|-------|
| mqttWriter.WriteUint16 | mqtt.go:5850 | PARTIAL | src/NATS.Server/Mqtt/MqttPacketWriter.cs:12-14 | Inline in Write method, not a standalone helper |
| mqttWriter.WriteString | mqtt.go:5855 | PORTED | src/NATS.Server/Mqtt/MqttPacketWriter.cs:8 | Added standalone UTF-8 length-prefixed `WriteString` helper |
| mqttWriter.WriteBytes | mqtt.go:5859 | PORTED | src/NATS.Server/Mqtt/MqttPacketWriter.cs:11 | Added standalone length-prefixed `WriteBytes` helper with uint16 size guard |
| mqttWriter.WriteVarInt | mqtt.go:5864 | PORTED | src/NATS.Server/Mqtt/MqttPacketWriter.cs:19-37 | EncodeRemainingLength implemented |
| newMQTTWriter | mqtt.go:5878 | PARTIAL | src/NATS.Server/Mqtt/MqttPacketWriter.cs:3 | Static class, no constructor needed; Write method serves the purpose |
---
## Keeping This File Updated
After porting work is completed:
1. **Update status**: Change `MISSING → PORTED` or `PARTIAL → PORTED` for each item completed
2. **Add .NET path**: Fill in the ".NET Equivalent" column with the actual file:line
3. **Re-count LOC**: Update the LOC numbers in `stillmissing.md`:
```bash
# Re-count .NET source LOC for this module
find src/NATS.Server/Mqtt/ -name '*.cs' -type f -exec cat {} + | wc -l
# Re-count .NET test LOC for this module
find tests/NATS.Server.Tests/Mqtt/ -name '*.cs' -type f -exec cat {} + | wc -l
```
4. **Add a changelog entry** below with date and summary of what was ported
5. **Update the parity DB** if new test mappings were created:
```bash
sqlite3 docs/test_parity.db "INSERT INTO test_mappings (go_test_id, dotnet_test_id, confidence, notes) VALUES (?, ?, 'manual', 'ported in YYYY-MM-DD session')"
```
## Summary Counts
| Status | Count |
|--------|-------|
| PORTED | 39 |
| PARTIAL | 57 |
| MISSING | 94 |
| NOT_APPLICABLE | 5 |
| DEFERRED | 0 |
| **Total** | **195** |
## Change Log
| Date | Change | By |
|------|--------|----|
| 2026-02-25 | Ported MQTT constants/writer parity batch: internal subject/stream constants, JSA reply token constants, Sparkplug constants, MQTT→NATS header constants, plus standalone `MqttPacketWriter.WriteString/WriteBytes` helpers with targeted tests (`MqttProtocolConstantsParityBatch2Tests`). | codex |
| 2026-02-26 | Ported MQTT helper-model parity batch: added missing helper/data models (`MqttJsa`, `MqttJsPubMsg`, `MqttRetMsgDel`, `MqttPersistedSession`, `MqttRetainedMessageRef`, `MqttSub`, `MqttFilter`, `MqttParsedPublishNatsHeader`) and extended retained message model with `Origin`/`Flags`/`Source`; verified in `MqttModelParityBatch3Tests`. | codex |
| 2026-02-25 | Full gap analysis completed: 195 items analyzed. 14 PORTED, 57 PARTIAL, 119 MISSING, 5 NOT_APPLICABLE. Major gaps: JetStream integration (entire mqttJSA layer ~30 functions), binary protocol encoding (CONNACK/SUBACK/UNSUBACK/PUBLISH serialization), delivery callbacks (QoS0/QoS1/QoS2), account session management, retained message encoding/decoding, Sparkplug B support, NATS subject reverse mapping. .NET has solid foundation for packet reading/writing, connect parsing, basic pub/sub flow, QoS tracking, and retained store, but all are simplified/in-memory-only without JetStream backing. | claude |
| 2026-02-25 | File created with LLM analysis instructions | auto |
Reference in New Issue View Git Blame Copy Permalink