natsnet/docs/plans/2026-02-27-batch-0-implementable-tests-plan.md

Batch 0 Implementable Tests Implementation Plan

For Codex: REQUIRED SUB-SKILL: Use executeplan to implement this plan task-by-task.

Goal: Port and verify all currently implementable Batch 0 deferred tests (553 candidates) whose feature dependencies are already verified, while keeping runtime-blocked tests deferred with explicit notes.

Architecture: Use a query-driven manifest as the Batch 0 source of truth, then execute class-by-class test porting in ImplBacklog files with tight red/green verification loops. After each class wave, update only the proven test IDs in PortTracker (verified for passing ports, deferred with notes for runtime-blocked tests). Keep production code unchanged unless a test reveals a real feature regression.

Tech Stack: .NET 10, xUnit 3, Shouldly, NSubstitute, SQLite (porting.db), PortTracker CLI

Design doc: docs/plans/2026-02-27-batch-0-implementable-tests-design.md

---

MANDATORY VERIFICATION PROTOCOL

This section is NON-NEGOTIABLE. Every task MUST follow this protocol. Skipping any step is a plan violation.

What Counts as a Real Test

A test is real (eligible for verified status) ONLY if ALL of these are true:

1. Has Arrange/Act/Assert — the method body contains setup, an invocation of production code, and at least one Shouldly assertion on the result
2. Calls production code — the test invokes at least one method from ZB.MOM.NatsNet.Server.* (not just test helpers or constants)
3. Has meaningful assertions — uses ShouldBe, ShouldContain, ShouldThrow, ShouldNotBeNull, etc. on values derived from the Act step
4. Is NOT a stub — does not contain throw new NotImplementedException, Assert.True(true), Assert.Pass(), // TODO, or an empty body

Stub Detection Check (REQUIRED after every class)

After porting all tests in a class, run this stub scan before running the tests:

# Scan for stub patterns — ANY match means the test is NOT real
grep -n -E "(NotImplementedException|Assert\.True\(true\)|Assert\.Pass|\.ShouldBe\(true\);$|// TODO|// PLACEHOLDER)" \
  dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/<ClassName>.Impltests.cs

If any matches are found: fix them or reclassify the test as deferred. Do NOT proceed to run tests until all stubs are eliminated.

Assertion Count Check (REQUIRED after every class)

# Count Shouldly assertions per test file — must average >= 1.5 per test method
grep -c "\.Should" dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/<ClassName>.Impltests.cs
grep -c "\[Fact\]\|\[Theory\]" dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/<ClassName>.Impltests.cs

If assertion count / test count < 1.5: review tests for shallow coverage. Tests with only one trivial assertion (e.g., result.ShouldNotBeNull()) on a complex method need strengthening.

Per-Test Verification Loop

For EVERY test method ported (not per-class, not per-wave — PER TEST):

1. Read the Go test — open the Go source at the line from test show <id> and understand what behavior it verifies
2. Write the C# port — translate the behavioral intent (not line-for-line)
3. Run the single test and capture output:

   dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
     --filter "FullyQualifiedName~ZB.MOM.NatsNet.Server.Tests.ImplBacklog.<ClassName>.<MethodName>" \
     --verbosity normal 2>&1 | tail -20
   

4. Verify the output shows Passed: 1, Failed: 0 — if it shows Passed: 0 the test was not discovered (name mismatch)
5. Only after green: add the test ID to the verified-candidates list for this class

Class-Level Gate (REQUIRED before any status updates)

After all tests in a class are individually verified:

1. Run the full class:

   dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
     --filter "FullyQualifiedName~ZB.MOM.NatsNet.Server.Tests.ImplBacklog.<ClassName>" \
     --verbosity normal 2>&1 | tail -5
   

2. Parse the summary line — extract Passed: N, Failed: M, Skipped: S
3. Verify: Failed MUST be 0, and Passed MUST equal the number of tests you ported
4. If mismatch: investigate before proceeding — do not mark anything verified

Status Update Protocol

• NEVER use test batch-update with more than 15 IDs at once — process in chunks of max 15
• NEVER mark a test verified unless you have captured and reviewed its individual pass output
• For each batch-update command, list the specific test IDs and the evidence (class run output showing pass count)
• Deferred tests MUST have a reason — when keeping a test deferred, update its notes with why (e.g., "requires running NATS server", "needs cluster topology")

Checkpoint Protocol (REQUIRED between tasks)

After completing each task (Tasks 2-9), before starting the next:

1. Run the full test suite to check for regressions:

   dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --verbosity normal 2>&1 | tail -10
   

2. Report the total pass/fail/skip counts
3. If any pre-existing tests broke, fix them before continuing
4. Commit the work for this task before starting the next

---

Task 1: Reconcile Batch 0 Mapping (DONE — commit ee0827d)

Batch 0 now has 553 tests mapped. This task is complete.

---

Task 2: Port Account/Auth/Options Cluster (89 tests)

Max scope: 7 classes, ~89 tests. Work ONE CLASS at a time.

Files:

• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/AccountTests.Impltests.cs (18 tests)
• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/AuthHandlerTests.Impltests.cs (3 tests)
• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/AuthCalloutTests.Impltests.cs (17 tests)
• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JwtProcessorTests.Impltests.cs (28 tests)
• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ServerOptionsTests.Impltests.cs (3 tests)
• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConfigReloaderTests.Impltests.cs (19 tests)
• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/CertificateStoreWindowsTests.Impltests.cs (1 test)

For EACH class, follow this sequence:

1. Read all Go test sources for the class (use test show <id> to get file+line for each test)
2. Port each test method following the Per-Test Verification Loop above
3. Run Stub Detection Check on the class file
4. Run Assertion Count Check on the class file
5. Run Class-Level Gate and capture output
6. Record verified IDs (only tests with individual + class-level green evidence)
7. Update status in chunks of max 15:

   dotnet run --project tools/NatsNet.PortTracker -- \
     test batch-update --ids "<max 15 comma-separated IDs>" --set-status verified --db porting.db --execute
   

8. For any test that requires live server/cluster infra, keep deferred and note the reason

After ALL 7 classes: run Checkpoint Protocol, then commit:

git add dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog porting.db
git commit -m "test(batch0): port account/auth/options implementable tests"

---

Task 3: Port Routing and Edge Transport Cluster (111 tests)

Max scope: 4 classes, ~111 tests. Work ONE CLASS at a time.

Files:

• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/RouteHandlerTests.Impltests.cs (25 tests)
• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/GatewayHandlerTests.Impltests.cs (21 tests)
• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/LeafNodeHandlerTests.Impltests.cs (47 tests)
• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/WebSocketHandlerTests.Impltests.cs (18 tests)

For EACH class: follow the same sequence as Task 2 (per-test loop, stub check, assertion check, class gate, chunked status updates).

After ALL 4 classes: run Checkpoint Protocol, then commit:

git add dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog porting.db
git commit -m "test(batch0): port routing/gateway/leaf/websocket implementable tests"

---

Task 4: Port Server Introspection and Events Cluster (128 tests)

Max scope: 4 classes, ~128 tests. Work ONE CLASS at a time.

Files:

• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/EventsHandlerTests.Impltests.cs (21 tests)
• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MonitoringHandlerTests.Impltests.cs (76 tests)
• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MessageTracerTests.Impltests.cs (20 tests)
• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/NatsServerTests.Impltests.cs (11 tests)

For EACH class: follow the same sequence as Task 2.

Special note for MonitoringHandlerTests (76 tests): This is the second-largest class. Process in sub-batches of ~20 tests. After every 20 tests, run the class-level gate to confirm cumulative pass count is growing and no regressions.

After ALL 4 classes: run Checkpoint Protocol, then commit:

git add dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog porting.db
git commit -m "test(batch0): port monitoring/events/server implementable tests"

---

Task 5: Port Concurrency Cluster (20 tests)

Max scope: 2 classes, ~20 tests. Work ONE CLASS at a time.

Files:

• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests1.Impltests.cs (18 tests)
• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests2.Impltests.cs (2 tests)

For EACH class: follow the same sequence as Task 2, with this addition:

Stability check — run each class 3 times to detect flaky tests:

for i in 1 2 3; do
  dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
    --filter "FullyQualifiedName~ZB.MOM.NatsNet.Server.Tests.ImplBacklog.<ClassName>" \
    --verbosity normal 2>&1 | tail -5
done

Only mark verified if all 3 runs pass with the same count.

After BOTH classes: run Checkpoint Protocol, then commit:

git add dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog porting.db
git commit -m "test(batch0): port concurrency implementable tests"

---

Task 6: Port JetStream Small/Deterministic Cluster (18 tests)

Max scope: 6 classes, ~18 tests. Work ONE CLASS at a time.

Files:

• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamVersioningTests.Impltests.cs (2 tests)
• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamBatchingTests.Impltests.cs (1 test)
• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamJwtTests.Impltests.cs (4 tests)
• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamTpmTests.Impltests.cs (5 tests)
• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamLeafNodeTests.Impltests.cs (6 tests)
• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/StorageEngineTests.Impltests.cs (5 tests — note: 5 already verified in prior work)

For EACH class: follow the same sequence as Task 2.

After ALL 6 classes: run Checkpoint Protocol, then commit:

git add dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog porting.db
git commit -m "test(batch0): port jetstream small/deterministic implementable tests"

---

Task 7: Port JetStream Consumer/Engine Heavy Cluster (125 tests)

Max scope: 3 classes, ~125 tests. Work ONE CLASS at a time.

Files:

• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/NatsConsumerTests.Impltests.cs (35 tests)
• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamEngineTests.Impltests.cs (89 tests)
• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests2.Impltests.cs (1 test)

For EACH class: follow the same sequence as Task 2.

Special note for JetStreamEngineTests (89 tests): This is the largest class. Process in sub-batches of ~15 tests. After every 15 tests:

• Run the stub detection check on the whole file
• Run the class-level gate and confirm cumulative pass count
• If pass count plateaus (same count after adding 15 tests), STOP — tests are likely stubs

After ALL 3 classes: run Checkpoint Protocol, then commit:

git add dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog porting.db
git commit -m "test(batch0): port jetstream consumer/engine implementable tests"

---

Task 8: Port MQTT Cluster (57 tests)

Max scope: 2 classes, ~57 tests. Work ONE CLASS at a time.

Files:

• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MqttHandlerTests.Impltests.cs (56 tests)
• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MqttExternalTests.Impltests.cs (1 test)

For EACH class: follow the same sequence as Task 2.

Special note for MqttHandlerTests (56 tests): Process in sub-batches of ~15 tests with stub detection after each sub-batch.

After BOTH classes: run Checkpoint Protocol, then commit:

git add dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog porting.db
git commit -m "test(batch0): port mqtt implementable tests"

---

Task 9: Final Batch 0 Verification and Status Closure

Files:

• Modify: porting.db
• Generate: reports/current.md (via report generation script)

Step 1: Full regression sweep

dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --verbosity normal 2>&1 | tail -10

Capture and report total pass/fail/skip. Failed MUST be 0.

Step 2: Stub audit across ALL ImplBacklog files

grep -rn -E "(NotImplementedException|Assert\.True\(true\)|Assert\.Pass|// TODO|// PLACEHOLDER)" \
  dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/

If any matches remain: fix them or reclassify the affected tests as deferred. Do NOT close the batch with stubs present.

Step 3: Validate remaining implementable deferred count

sqlite3 porting.db "
WITH implementable AS (
  SELECT t.id
  FROM unit_tests t
  WHERE t.status='deferred'
    AND EXISTS (
      SELECT 1 FROM dependencies d
      JOIN features f ON f.id=d.target_id AND d.target_type='feature'
      WHERE d.source_type='unit_test' AND d.source_id=t.id
    )
    AND NOT EXISTS (
      SELECT 1 FROM dependencies d
      JOIN features f ON f.id=d.target_id AND d.target_type='feature'
      WHERE d.source_type='unit_test' AND d.source_id=t.id
        AND f.status NOT IN ('verified','complete','n_a')
    )
)
SELECT COUNT(*) FROM implementable;
"

Expected: 0 (or only explicitly documented runtime-blocked exceptions with notes).

Step 4: Verify Batch 0 visibility

dotnet run --project tools/NatsNet.PortTracker -- batch show 0 --db porting.db
dotnet run --project tools/NatsNet.PortTracker -- report summary --db porting.db

Step 5: Generate report and final commit

./reports/generate-report.sh
git add dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog porting.db reports/
git commit -m "test(batch0): complete batch 0 implementable tests verification"

---

Batch 0 Working Set (Current)

• Total implementable candidates: 553
• Primary files: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/*.Impltests.cs
• Highest-volume classes:
  • JetStreamEngineTests (89)
  • MonitoringHandlerTests (76)
  • MqttHandlerTests (56)
  • LeafNodeHandlerTests (47)
  • NatsConsumerTests (35)

ANTI-STUB GUARDRAILS (NON-NEGOTIABLE)

These rules exist because previous attempts degraded into stub-writing after ~50 tests. Every rule is mandatory.

Forbidden Patterns

The following patterns in a test body mean the test is a STUB, not a real port. Tests containing these MUST NOT be marked verified:

Pattern	Why it's a stub
throw new NotImplementedException()	Placeholder, tests nothing
Assert.True(true)	Always passes, tests nothing
Assert.Pass()	Always passes, tests nothing
result.ShouldBe(true); (alone)	Likely a lazy replacement for real assertion
// TODO or // PLACEHOLDER in body	Incomplete work marker
Empty method body { }	Tests nothing
Only ShouldNotBeNull() on a constructor	Trivial, not behavioral

Verification Evidence Requirements

Before marking ANY test as verified, you MUST have:

1. Individual test output showing Passed: 1 for that specific test method
2. Class-level output showing total Passed: N matches the number of tests you ported in that class
3. Stub scan output showing zero matches for the class file
4. Assertion count showing >= 1.5 assertions per test on average

Hard Limits

• Max 15 test IDs per batch-update call — forces you to think about what you're marking
• Max 1 class per status-update cycle — no cross-class bulk updates
• Mandatory commit after each task — prevents loss of work and creates review points
• If pass count plateaus after adding tests, STOP and investigate — this means tests are stubs or not being discovered

If You Get Stuck

If a test requires infrastructure you can't provide (running server, cluster, etc.):

1. Keep the test as deferred — do NOT write a stub that trivially passes
2. Add a comment in the test: // DEFERRED: requires <specific reason>
3. Move on to the next test
4. This is the CORRECT behavior — marking deferred with a reason is better than a fake pass