natsnet/docs/plans/2026-02-27-batch-41-mqtt-client-io-implementation-plan.md

Batch 41 MQTT Client/IO Implementation Plan

For Codex: REQUIRED SUB-SKILL: Use executeplan to implement this plan task-by-task.

Goal: Port and verify Batch 41 (MQTT Client/IO) so all 74 mapped features and 28 mapped tests are either truly implemented and verified or explicitly deferred with concrete blocker reasons.

Architecture: Implement in four feature groups (19/19/17/19) aligned to connect/publish/session handling, retained and QoS flow, subscription lifecycle, and reader/writer conversion I/O. Execute test waves after each group with strict evidence gates and explicit deferred handling for runtime-blocked cross-module tests.

Tech Stack: .NET 10, C# latest, xUnit 3, Shouldly, NSubstitute, PortTracker CLI, SQLite (porting.db)

Design doc: docs/plans/2026-02-27-batch-41-mqtt-client-io-design.md

---

Batch 41 Scope

• Batch ID: 41
• Name: MQTT Client/IO
• Dependency: Batch 40
• Go source: golang/nats-server/server/mqtt.go
• Feature IDs: 2331-2404 (74)
• Test IDs: 115,1055,1056,1113,1258,1924,2170,2171,2182,2190,2191,2194,2195,2196,2199,2200,2204,2229,2234,2235,2236,2237,2238,2246,2251,2253,2285,3095

Use this runtime path if dotnet is unavailable on PATH:

DOTNET=/usr/local/share/dotnet/dotnet

Expected source files (create/modify as needed):

• dotnet/src/ZB.MOM.NatsNet.Server/Mqtt/MqttHandler.cs
• dotnet/src/ZB.MOM.NatsNet.Server/Mqtt/MqttTypes.cs
• dotnet/src/ZB.MOM.NatsNet.Server/Mqtt/MqttConstants.cs
• dotnet/src/ZB.MOM.NatsNet.Server/ClientConnection*.cs (MQTT partials)
• dotnet/src/ZB.MOM.NatsNet.Server/NatsServer*.cs (MQTT partials)
• dotnet/src/ZB.MOM.NatsNet.Server/Mqtt/MqttReader*.cs (new/partial)
• dotnet/src/ZB.MOM.NatsNet.Server/Mqtt/MqttWriter*.cs (new/partial)
• dotnet/src/ZB.MOM.NatsNet.Server/Mqtt/MqttSession*.cs (new/partial)

Expected test files (create/modify as needed):

• dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MqttHandlerTests.Impltests.cs
• dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MqttExternalTests.Impltests.cs
• dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/AuthCalloutTests.Impltests.cs
• dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/NatsConsumerTests.Impltests.cs
• dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/LeafNodeHandlerTests.Impltests.cs
• dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/WebSocketHandlerTests.Impltests.cs
• dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests3.Impltests.cs (create if missing)

---

MANDATORY VERIFICATION PROTOCOL

NON-NEGOTIABLE: Every Batch 41 feature/test item must pass this protocol before status promotion.

1. Dependency and Baseline Preflight (REQUIRED)

Before touching any status:

$DOTNET run --project tools/NatsNet.PortTracker -- batch show 40 --db porting.db
$DOTNET run --project tools/NatsNet.PortTracker -- batch show 41 --db porting.db
$DOTNET run --project tools/NatsNet.PortTracker -- batch ready --db porting.db

Start only when Batch 41 is ready:

$DOTNET run --project tools/NatsNet.PortTracker -- batch start 41 --db porting.db

Capture baseline:

$DOTNET build dotnet/
$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/

2. Per-Feature Verification Loop (REQUIRED per feature ID)

For each feature ID in active group:

1. Inspect mapping and Go behavior:

   $DOTNET run --project tools/NatsNet.PortTracker -- feature show <feature_id> --db porting.db
   

2. Claim the feature:

   $DOTNET run --project tools/NatsNet.PortTracker -- feature update <feature_id> --status stub --db porting.db
   

3. Implement minimal correct logic in mapped .NET class/method.
4. Add or update at least one behavioral test path covering this feature.
5. Run stub detection checks (source + test).
6. Run build gate.
7. Run targeted test gate for touched classes/methods.
8. Promote to complete only when gates pass.
9. Promote to verified only at task checkpoint.

3. Per-Test Verification Loop (REQUIRED per test ID)

For each mapped test ID in active wave:

1. Inspect mapped test details:

   $DOTNET run --project tools/NatsNet.PortTracker -- test show <test_id> --db porting.db
   

2. Claim the test:

   $DOTNET run --project tools/NatsNet.PortTracker -- test update <test_id> --status stub --db porting.db
   

3. Port full Arrange/Act/Assert behavior (no placeholders).
4. Run single-test filter and verify discovery:

   $DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
     --filter "FullyQualifiedName~<ClassName>.<MethodName>" --verbosity normal
   

5. Confirm output includes Passed: 1, Failed: 0 (never accept Passed: 0).
6. Run class-level test filter.
7. Run stub detection + build gate.
8. Promote to complete; promote verified only at checkpoint.

4. Stub Detection Check (REQUIRED)

Run after each feature loop, each test loop, and before any status update.

Source stub scan:

src_changed=$(git diff --name-only -- dotnet/src/ZB.MOM.NatsNet.Server | rg "\\.cs$" || true)
if [ -n "$src_changed" ]; then
  echo "$src_changed" | xargs rg -n "(NotImplementedException|// TODO|// PLACEHOLDER|=>\\s*default;|return\\s+default;|return\\s+null;|throw\\s+new\\s+NotImplementedException)"
fi

Test stub scan:

test_changed=$(git diff --name-only -- dotnet/tests/ZB.MOM.NatsNet.Server.Tests | rg "\\.cs$" || true)
if [ -n "$test_changed" ]; then
  echo "$test_changed" | xargs rg -n "(NotImplementedException|Assert\\.True\\(true\\)|Assert\\.Pass\\(\\)|// TODO|// PLACEHOLDER|goFile\\s*=\\s*\"server/|ShouldContain\\(\"Should\"\\)|ShouldBe\\(string\\.Empty\\))"
fi

Assertion depth check for changed test files:

# per file: Shouldly assertion count / test count should be >= 1.5

5. Build Gate (REQUIRED)

$DOTNET build dotnet/

Run after every feature/test loop, before any batch-update, and at every checkpoint.

6. Test Gate (REQUIRED)

Run minimal relevant filters continuously, and full suite at checkpoints.

MQTT core filters:

$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
  --filter "FullyQualifiedName~ImplBacklog.MqttHandlerTests|FullyQualifiedName~ImplBacklog.MqttExternalTests"

Cross-module mapped filters:

$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
  --filter "FullyQualifiedName~ImplBacklog.AuthCalloutTests|FullyQualifiedName~ImplBacklog.NatsConsumerTests|FullyQualifiedName~ImplBacklog.LeafNodeHandlerTests|FullyQualifiedName~ImplBacklog.WebSocketHandlerTests|FullyQualifiedName~ImplBacklog.JetStreamClusterTests3"

Checkpoint full unit gate:

$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/

7. Status Update Protocol (HARD LIMIT: 15 IDs)

• Maximum 15 IDs per feature batch-update call.
• Maximum 15 IDs per test batch-update call.
• Never promote verified without checkpoint evidence.
• Deferred items must include explicit blocker reason.
• Status updates are scoped to the active task only.

Commands:

$DOTNET run --project tools/NatsNet.PortTracker -- \
  feature batch-update --ids "<max 15 ids>" --set-status <stub|complete|verified> --db porting.db --execute

$DOTNET run --project tools/NatsNet.PortTracker -- \
  test batch-update --ids "<max 15 ids>" --set-status <stub|complete|verified> --db porting.db --execute

8. Checkpoint Protocol Between Tasks (REQUIRED)

Before starting next task:

1. Run source + test stub detection scans.
2. Run build gate.
3. Run targeted test filters plus full unit test suite.
4. Apply status updates in <=15-ID chunks.
5. Run batch progress check:

   $DOTNET run --project tools/NatsNet.PortTracker -- batch show 41 --db porting.db
   

6. Commit checkpoint before moving on.

---

ANTI-STUB GUARDRAILS

NON-NEGOTIABLE: Placeholder behavior is not valid progress.

Forbidden Patterns

Any of these in mapped feature/test bodies blocks status promotion:

• throw new NotImplementedException(...)
• Empty mapped method body
• // TODO or // PLACEHOLDER left in mapped logic
• => default;, return default;, return null; in mapped feature methods
• Assert.True(true) or Assert.Pass()
• Placeholder template assertions not tied to behavior:
  • var goFile = "server/..." as test core
  • "...".ShouldContain("Should")
  • GetRequiredApiLevel(new Dictionary<string, string>()).ShouldBe(string.Empty) as primary assertion
• Test methods with only trivial constructor/null checks for complex protocol behavior

Hard Limits

• Feature groups must stay <=20 IDs.
• Max 15 IDs per status batch-update command.
• One active feature loop at a time.
• One active test loop at a time.
• Mandatory checkpoint between tasks.
• No cross-task bulk status updates.
• No verified promotion without checkpoint evidence.

If You Get Stuck (REQUIRED)

If an item needs infra or dependencies not available:

1. Stop work on that ID.
2. Do not add placeholders or force a pass.
3. Mark deferred with exact blocker reason.
4. Continue with the next unblocked ID.

Feature deferral:

$DOTNET run --project tools/NatsNet.PortTracker -- \
  feature update <feature_id> --status deferred --override "blocked: <specific reason>" --db porting.db

Test deferral:

$DOTNET run --project tools/NatsNet.PortTracker -- \
  test update <test_id> --status deferred --override "blocked: <specific reason>" --db porting.db

deferred with concrete reason is correct behavior. Stubs are not.

---

Feature Groups (max ~20 each)

Group A (19): session, CONNECT, publish ingress

IDs: 2331,2332,2333,2334,2335,2336,2337,2338,2339,2340,2341,2342,2343,2344,2345,2346,2347,2348,2349

Group B (19): retained permissions, QoS ack flow, subscribe callbacks

IDs: 2350,2351,2352,2353,2354,2355,2356,2357,2358,2359,2360,2361,2362,2363,2364,2365,2366,2367,2368

Group C (17): reserved-sub/sparkplug, subscribe/unsubscribe processing

IDs: 2369,2370,2371,2372,2373,2374,2375,2376,2377,2378,2379,2380,2381,2382,2383,2384,2385

Group D (19): subject conversion and reader/writer I/O

IDs: 2386,2387,2388,2389,2390,2391,2392,2393,2394,2395,2396,2397,2398,2399,2400,2401,2402,2403,2404

Test Waves (28 total)

• Wave T1 (10): 2170,2171,2190,2191,2194,2195,2196,2199,2200,2229
• Wave T2 (11): 2182,2204,2234,2235,2236,2237,2238,2246,2251,2253,2285
• Wave T3 (4): 115,1258,1924,3095
• Wave T4 (3): 1055,1056,1113

---

Task 1: Preflight, Mapping Alignment, and Baseline

Files:

• Read: docs/plans/2026-02-27-batch-41-mqtt-client-io-design.md
• Read: golang/nats-server/server/mqtt.go
• Modify/Create: Batch 41 mapped class/method surfaces under dotnet/src/ZB.MOM.NatsNet.Server/Mqtt/, dotnet/src/ZB.MOM.NatsNet.Server/ClientConnection*.cs, and dotnet/src/ZB.MOM.NatsNet.Server/NatsServer*.cs

Steps:

1. Verify Batch 40 completion/readiness and start Batch 41.
2. Ensure all mapped Batch 41 methods exist and compile.
3. Capture baseline build/test state.
4. Run checkpoint protocol.

Task 2: Implement Group A + Wave T1 (Parser/Connect Core)

Files:

• Modify/Create: ClientConnection MQTT parse/connect files
• Modify/Create: NatsServer MQTT connect/publish ingress files
• Modify: dotnet/src/ZB.MOM.NatsNet.Server/Mqtt/MqttHandler.cs
• Modify tests: ImplBacklog/MqttHandlerTests.Impltests.cs, ImplBacklog/MqttExternalTests.Impltests.cs

Steps:

1. Process all Group A feature IDs through per-feature loop.
2. Port/verify T1 parser/conversion tests through per-test loop.
3. Apply status updates in <=15-ID chunks.
4. Run checkpoint protocol and commit.

Task 3: Implement Group B + Wave T2 (Retain/QoS Core)

Files:

• Modify/Create: MQTT retained/perms and QoS ack handling files
• Modify tests: ImplBacklog/MqttHandlerTests.Impltests.cs

Steps:

1. Process Group B features through per-feature loop.
2. Port/verify T2 tests tied to retain, QoS flags, pub ack flow.
3. Defer blocked IDs with explicit reasons (no stubs).
4. Apply <=15-ID status updates.
5. Run checkpoint protocol and commit.

Task 4: Implement Group C + Remaining T2/T3 (Subscription Lifecycle)

Files:

• Modify/Create: MQTT subscription/publish delivery/session consumer files
• Modify tests: ImplBacklog/MqttHandlerTests.Impltests.cs, ImplBacklog/AuthCalloutTests.Impltests.cs, ImplBacklog/NatsConsumerTests.Impltests.cs, ImplBacklog/LeafNodeHandlerTests.Impltests.cs, ImplBacklog/WebSocketHandlerTests.Impltests.cs

Steps:

1. Process Group C features through per-feature loop.
2. Complete remaining T2 tests and attempt T3 cross-module tests.
3. For cross-module blockers, defer with specific dependency reason.
4. Apply <=15-ID status updates.
5. Run checkpoint protocol and commit.

Task 5: Implement Group D + T4 Cluster-Mapped Tests (Reader/Writer + Conversion)

Files:

• Modify/Create: MqttReader*, MqttWriter*, conversion helper files
• Modify tests: ImplBacklog/MqttHandlerTests.Impltests.cs, ImplBacklog/JetStreamClusterTests3.Impltests.cs (create if absent)

Steps:

1. Process Group D features through per-feature loop.
2. Port/verify reader/writer conversion tests first.
3. Attempt T4 cluster tests only where dependencies allow; otherwise defer explicitly.
4. Apply <=15-ID status updates.
5. Run checkpoint protocol and commit.

Task 6: Final Batch 41 Verification and Closure

Files:

• Modify: porting.db
• Generate: reports/current.md (via report script)

Steps:

1. Run final source/test stub scans on all changed files.
2. Run build and full unit test suite.
3. Verify Batch 41 status:

   $DOTNET run --project tools/NatsNet.PortTracker -- batch show 41 --db porting.db
   $DOTNET run --project tools/NatsNet.PortTracker -- report summary --db porting.db
   

4. Generate report:

   ./reports/generate-report.sh
   

5. Commit final checkpoint.