Files

Joseph Doherty c7bbf45c8f docs: add remaining jetstream parity plan

2026-02-23 09:51:21 -05:00

40 KiB

Raw Blame History

JetStream Remaining Parity Implementation Plan

For Codex: REQUIRED SUB-SKILL: Use executeplan to implement this plan task-by-task.

Goal: Port all remaining JetStream features from Go to .NET, including missing API subjects, runtime semantics, storage/recovery behavior, and cluster/RAFT control operations, then update parity evidence and differences.md.

Architecture: Execute in behavior-core-first slices. First create an explicit parity inventory and API matrix, then complete runtime/state semantics for streams and consumers, then wire remaining $JS.API.* families to those semantics. Finish with storage/RAFT/monitoring parity, dual-gate verification evidence, and differences.md updates.

Tech Stack: .NET 10, C# 14, xUnit 3, Shouldly, NSubstitute, NATS.NKeys, Serilog, bash scripts, ripgrep.

Execution guardrails

Use @test-driven-development in every task.
Use @systematic-debugging if observed behavior diverges from test intent.
Run @verification-before-completion before any success claim.
Perform this plan in a dedicated worktree to keep parity work isolated.

Task 1: Build Remaining Feature Inventory and API Matrix

Files:

Create: scripts/jetstream/extract-go-js-api.sh
Create: docs/plans/2026-02-23-jetstream-remaining-parity-map.md
Test: tests/NATS.Server.Tests/JetStreamApiInventoryTests.cs

Step 1: Write the failing test

[Fact]
public void Go_inventory_contains_api_subjects_not_yet_mapped_in_dotnet()
{
    var inventory = JetStreamApiInventory.LoadFromGoConstants();
    inventory.GoSubjects.ShouldContain("$JS.API.STREAM.UPDATE.*");
    inventory.GoSubjects.ShouldContain("$JS.API.CONSUMER.MSG.NEXT.*.*");
    inventory.GoSubjects.Count.ShouldBeGreaterThan(20);
}

Step 2: Run test to verify it fails

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamApiInventoryTests.Go_inventory_contains_api_subjects_not_yet_mapped_in_dotnet" -v minimal
Expected: FAIL with missing inventory loader/script output.

Step 3: Write minimal implementation

#!/usr/bin/env bash
set -euo pipefail
rg -n -F '$JS.API' golang/nats-server/server/jetstream_api.go \
  | awk -F: '{print $3}' \
  | sed -E 's/.*"(\$JS\.API[^\"]+)".*/\1/' \
  | sort -u

# JetStream Remaining Parity Map

| Go Subject | .NET Route | Status | Test |
|---|---|---|---|
| $JS.API.STREAM.UPDATE.* | (pending) | pending | (pending) |

Step 4: Run test to verify it passes

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamApiInventoryTests" -v minimal
Expected: PASS.

Step 5: Commit

git add scripts/jetstream/extract-go-js-api.sh docs/plans/2026-02-23-jetstream-remaining-parity-map.md tests/NATS.Server.Tests/JetStreamApiInventoryTests.cs
git commit -m "test: add jetstream remaining api inventory and parity map scaffold"

Task 2: Wire `$JS.API.*` Request/Reply Through NATS Message Path

Files:

Modify: src/NATS.Server/NatsServer.cs
Modify: src/NATS.Server/NatsClient.cs
Test: tests/NATS.Server.Tests/JetStreamApiProtocolIntegrationTests.cs

Step 1: Write the failing test

[Fact]
public async Task Js_api_request_over_pub_reply_returns_response_message()
{
    await using var server = await ServerFixture.StartJetStreamEnabledAsync();
    var response = await server.RequestAsync("$JS.API.INFO", "{}", timeoutMs: 1000);

    response.ShouldContain("\"error\"");
}

Step 2: Run test to verify it fails

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamApiProtocolIntegrationTests.Js_api_request_over_pub_reply_returns_response_message" -v minimal
Expected: FAIL because $JS.API.* is not processed in protocol request path.

Step 3: Write minimal implementation

if (replyTo != null && subject.StartsWith("$JS.API", StringComparison.Ordinal) && _jetStreamApiRouter != null)
{
    var api = _jetStreamApiRouter.Route(subject, payload.Span);
    sender.SendJetStreamApiResponse(replyTo, api);
    return;
}

Step 4: Run test to verify it passes

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamApiProtocolIntegrationTests" -v minimal
Expected: PASS.

Step 5: Commit

git add src/NATS.Server/NatsServer.cs src/NATS.Server/NatsClient.cs tests/NATS.Server.Tests/JetStreamApiProtocolIntegrationTests.cs
git commit -m "feat: route jetstream api requests over pub reply path"

Task 3: Expand API Router to Full Subject Family Dispatcher

Files:

Create: src/NATS.Server/JetStream/Api/JetStreamApiSubjects.cs
Modify: src/NATS.Server/JetStream/Api/JetStreamApiRouter.cs
Test: tests/NATS.Server.Tests/JetStreamApiRouterCoverageTests.cs

Step 1: Write the failing test

[Theory]
[InlineData("$JS.API.STREAM.UPDATE.ORDERS")]
[InlineData("$JS.API.STREAM.DELETE.ORDERS")]
[InlineData("$JS.API.STREAM.PURGE.ORDERS")]
[InlineData("$JS.API.CONSUMER.DELETE.ORDERS.DUR")]
[InlineData("$JS.API.CONSUMER.MSG.NEXT.ORDERS.DUR")]
public void Router_recognizes_remaining_subject_families(string subject)
{
    var router = new JetStreamApiRouter(new StreamManager(), new ConsumerManager());
    var response = router.Route(subject, "{}"u8);
    response.ShouldNotBeNull();
}

Step 2: Run test to verify it fails

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamApiRouterCoverageTests" -v minimal
Expected: FAIL with NotFound for unhandled families.

Step 3: Write minimal implementation

public static class JetStreamApiSubjects
{
    public const string StreamUpdate = "$JS.API.STREAM.UPDATE.";
    public const string StreamDelete = "$JS.API.STREAM.DELETE.";
    public const string ConsumerDelete = "$JS.API.CONSUMER.DELETE.";
    public const string ConsumerNext = "$JS.API.CONSUMER.MSG.NEXT.";
}

if (subject.StartsWith(JetStreamApiSubjects.StreamUpdate, StringComparison.Ordinal))
    return StreamApiHandlers.HandleUpdate(subject, payload, _streamManager);

Step 4: Run test to verify it passes

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamApiRouterCoverageTests" -v minimal
Expected: PASS.

Step 5: Commit

git add src/NATS.Server/JetStream/Api/JetStreamApiSubjects.cs src/NATS.Server/JetStream/Api/JetStreamApiRouter.cs tests/NATS.Server.Tests/JetStreamApiRouterCoverageTests.cs
git commit -m "feat: expand jetstream api router subject families"

Task 4: Implement Account Info and System-Level API Responses

Files:

Create: src/NATS.Server/JetStream/Api/Handlers/AccountApiHandlers.cs
Modify: src/NATS.Server/JetStream/Api/JetStreamApiResponse.cs
Modify: src/NATS.Server/JetStream/Api/JetStreamApiRouter.cs
Test: tests/NATS.Server.Tests/JetStreamAccountInfoApiTests.cs

Step 1: Write the failing test

[Fact]
public void Account_info_returns_jetstream_limits_and_usage_shape()
{
    var router = new JetStreamApiRouter(new StreamManager(), new ConsumerManager());
    var response = router.Route("$JS.API.INFO", "{}"u8);

    response.AccountInfo.ShouldNotBeNull();
    response.Error.ShouldBeNull();
}

Step 2: Run test to verify it fails

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamAccountInfoApiTests" -v minimal
Expected: FAIL because $JS.API.INFO route is missing.

Step 3: Write minimal implementation

public static JetStreamApiResponse HandleInfo(StreamManager streams, ConsumerManager consumers)
{
    return new JetStreamApiResponse
    {
        AccountInfo = new JetStreamAccountInfo
        {
            Streams = streams.StreamNames.Count,
            Consumers = consumers.ConsumerCount,
        },
    };
}

Step 4: Run test to verify it passes

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamAccountInfoApiTests" -v minimal
Expected: PASS.

Step 5: Commit

git add src/NATS.Server/JetStream/Api/Handlers/AccountApiHandlers.cs src/NATS.Server/JetStream/Api/JetStreamApiResponse.cs src/NATS.Server/JetStream/Api/JetStreamApiRouter.cs tests/NATS.Server.Tests/JetStreamAccountInfoApiTests.cs
git commit -m "feat: add jetstream account info api response"

Task 5: Complete Stream Lifecycle APIs (Update/Delete)

Files:

Modify: src/NATS.Server/JetStream/Api/Handlers/StreamApiHandlers.cs
Modify: src/NATS.Server/JetStream/StreamManager.cs
Test: tests/NATS.Server.Tests/JetStreamStreamLifecycleApiTests.cs

Step 1: Write the failing test

[Fact]
public async Task Stream_update_and_delete_roundtrip()
{
    await using var fx = await JetStreamApiFixture.StartWithStreamAsync("ORDERS", "orders.*");

    var update = await fx.RequestLocalAsync("$JS.API.STREAM.UPDATE.ORDERS", "{\"subjects\":[\"orders.v2.*\"]}");
    update.Error.ShouldBeNull();

    var delete = await fx.RequestLocalAsync("$JS.API.STREAM.DELETE.ORDERS", "{}");
    delete.Success.ShouldBeTrue();
}

Step 2: Run test to verify it fails

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamStreamLifecycleApiTests" -v minimal
Expected: FAIL due to unimplemented update/delete handlers.

Step 3: Write minimal implementation

public JetStreamApiResponse Delete(string name)
{
    return _streams.TryRemove(name, out _) ? JetStreamApiResponse.SuccessResponse() : JetStreamApiResponse.NotFound($"$JS.API.STREAM.DELETE.{name}");
}

if (subject.StartsWith(JetStreamApiSubjects.StreamDelete, StringComparison.Ordinal))
    return StreamApiHandlers.HandleDelete(subject, _streamManager);

Step 4: Run test to verify it passes

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamStreamLifecycleApiTests" -v minimal
Expected: PASS.

Step 5: Commit

git add src/NATS.Server/JetStream/Api/Handlers/StreamApiHandlers.cs src/NATS.Server/JetStream/StreamManager.cs tests/NATS.Server.Tests/JetStreamStreamLifecycleApiTests.cs
git commit -m "feat: implement jetstream stream update and delete apis"

Task 6: Implement Stream Names/List APIs

Files:

Modify: src/NATS.Server/JetStream/Api/Handlers/StreamApiHandlers.cs
Modify: src/NATS.Server/JetStream/Api/JetStreamApiResponse.cs
Test: tests/NATS.Server.Tests/JetStreamStreamListApiTests.cs

Step 1: Write the failing test

[Fact]
public async Task Stream_names_and_list_return_created_streams()
{
    await using var fx = await JetStreamApiFixture.StartWithStreamAsync("ORDERS", "orders.*");
    _ = await fx.RequestLocalAsync("$JS.API.STREAM.CREATE.INVOICES", "{\"subjects\":[\"invoices.*\"]}");

    var names = await fx.RequestLocalAsync("$JS.API.STREAM.NAMES", "{}");
    names.StreamNames.ShouldContain("ORDERS");
    names.StreamNames.ShouldContain("INVOICES");
}

Step 2: Run test to verify it fails

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamStreamListApiTests" -v minimal
Expected: FAIL because list/names endpoints are missing.

Step 3: Write minimal implementation

public JetStreamApiResponse ListNames()
    => new() { StreamNames = [.. _streams.Keys.OrderBy(x => x, StringComparer.Ordinal)] };

Step 4: Run test to verify it passes

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamStreamListApiTests" -v minimal
Expected: PASS.

Step 5: Commit

git add src/NATS.Server/JetStream/Api/Handlers/StreamApiHandlers.cs src/NATS.Server/JetStream/Api/JetStreamApiResponse.cs tests/NATS.Server.Tests/JetStreamStreamListApiTests.cs
git commit -m "feat: add jetstream stream names and list apis"

Task 7: Implement Stream Purge and Message Delete/Get APIs

Files:

Modify: src/NATS.Server/JetStream/Api/Handlers/StreamApiHandlers.cs
Modify: src/NATS.Server/JetStream/Storage/IStreamStore.cs
Modify: src/NATS.Server/JetStream/Storage/MemStore.cs
Modify: src/NATS.Server/JetStream/Storage/FileStore.cs
Test: tests/NATS.Server.Tests/JetStreamStreamMessageApiTests.cs

Step 1: Write the failing test

[Fact]
public async Task Stream_msg_get_delete_and_purge_change_state()
{
    await using var fx = await JetStreamApiFixture.StartWithStreamAsync("ORDERS", "orders.*");
    var ack = await fx.PublishAndGetAckAsync("orders.created", "1");

    var get = await fx.RequestLocalAsync("$JS.API.STREAM.MSG.GET.ORDERS", $"{{\"seq\":{ack.Seq}}}");
    get.StreamMessage.ShouldNotBeNull();

    var del = await fx.RequestLocalAsync("$JS.API.STREAM.MSG.DELETE.ORDERS", $"{{\"seq\":{ack.Seq}}}");
    del.Success.ShouldBeTrue();

    var purge = await fx.RequestLocalAsync("$JS.API.STREAM.PURGE.ORDERS", "{}");
    purge.Success.ShouldBeTrue();
}

Step 2: Run test to verify it fails

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamStreamMessageApiTests" -v minimal
Expected: FAIL due to missing store operations and handlers.

Step 3: Write minimal implementation

public interface IStreamStore
{
    ValueTask<bool> RemoveAsync(ulong sequence, CancellationToken ct);
    ValueTask<IReadOnlyList<ulong>> PurgeAsync(CancellationToken ct);
}

public JetStreamApiResponse DeleteMessage(string stream, ulong seq)
    => _streamManager.DeleteMessage(stream, seq) ? JetStreamApiResponse.SuccessResponse() : JetStreamApiResponse.ErrorResponse(404, "message not found");

Step 4: Run test to verify it passes

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamStreamMessageApiTests" -v minimal
Expected: PASS.

Step 5: Commit

git add src/NATS.Server/JetStream/Api/Handlers/StreamApiHandlers.cs src/NATS.Server/JetStream/Storage/IStreamStore.cs src/NATS.Server/JetStream/Storage/MemStore.cs src/NATS.Server/JetStream/Storage/FileStore.cs tests/NATS.Server.Tests/JetStreamStreamMessageApiTests.cs
git commit -m "feat: add stream purge and message get delete apis"

Task 8: Implement Direct Message Get APIs

Files:

Create: src/NATS.Server/JetStream/Api/Handlers/DirectApiHandlers.cs
Modify: src/NATS.Server/JetStream/Api/JetStreamApiRouter.cs
Test: tests/NATS.Server.Tests/JetStreamDirectGetApiTests.cs

Step 1: Write the failing test

[Fact]
public async Task Direct_get_returns_message_without_stream_info_wrapper()
{
    await using var fx = await JetStreamApiFixture.StartWithStreamAsync("ORDERS", "orders.*");
    var ack = await fx.PublishAndGetAckAsync("orders.created", "1");

    var direct = await fx.RequestLocalAsync("$JS.API.DIRECT.GET.ORDERS", $"{{\"seq\":{ack.Seq}}}");
    direct.DirectMessage.ShouldNotBeNull();
    direct.DirectMessage!.Payload.ShouldBe("1");
}

Step 2: Run test to verify it fails

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamDirectGetApiTests" -v minimal
Expected: FAIL because direct get route does not exist.

Step 3: Write minimal implementation

if (subject.StartsWith("$JS.API.DIRECT.GET.", StringComparison.Ordinal))
    return DirectApiHandlers.HandleGet(subject, payload, _streamManager);

Step 4: Run test to verify it passes

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamDirectGetApiTests" -v minimal
Expected: PASS.

Step 5: Commit

git add src/NATS.Server/JetStream/Api/Handlers/DirectApiHandlers.cs src/NATS.Server/JetStream/Api/JetStreamApiRouter.cs tests/NATS.Server.Tests/JetStreamDirectGetApiTests.cs
git commit -m "feat: add jetstream direct get api"

Task 9: Implement Stream Snapshot/Restore API Skeleton with Store Hooks

Files:

Create: src/NATS.Server/JetStream/Snapshots/StreamSnapshotService.cs
Modify: src/NATS.Server/JetStream/Api/Handlers/StreamApiHandlers.cs
Modify: src/NATS.Server/JetStream/Storage/IStreamStore.cs
Test: tests/NATS.Server.Tests/JetStreamSnapshotRestoreApiTests.cs

Step 1: Write the failing test

[Fact]
public async Task Snapshot_then_restore_reconstructs_messages()
{
    await using var fx = await JetStreamApiFixture.StartWithStreamAsync("ORDERS", "orders.*");
    _ = await fx.PublishAndGetAckAsync("orders.created", "1");

    var snap = await fx.RequestLocalAsync("$JS.API.STREAM.SNAPSHOT.ORDERS", "{}");
    snap.Snapshot.ShouldNotBeNull();

    var restore = await fx.RequestLocalAsync("$JS.API.STREAM.RESTORE.ORDERS", snap.Snapshot!.Payload);
    restore.Success.ShouldBeTrue();
}

Step 2: Run test to verify it fails

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamSnapshotRestoreApiTests" -v minimal
Expected: FAIL due to missing snapshot/restore handlers and store contracts.

Step 3: Write minimal implementation

public interface IStreamStore
{
    ValueTask<byte[]> CreateSnapshotAsync(CancellationToken ct);
    ValueTask RestoreSnapshotAsync(ReadOnlyMemory<byte> snapshot, CancellationToken ct);
}

public sealed class StreamSnapshotService
{
    public ValueTask<byte[]> SnapshotAsync(StreamHandle stream, CancellationToken ct) => stream.Store.CreateSnapshotAsync(ct);
}

Step 4: Run test to verify it passes

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamSnapshotRestoreApiTests" -v minimal
Expected: PASS.

Step 5: Commit

git add src/NATS.Server/JetStream/Snapshots/StreamSnapshotService.cs src/NATS.Server/JetStream/Api/Handlers/StreamApiHandlers.cs src/NATS.Server/JetStream/Storage/IStreamStore.cs tests/NATS.Server.Tests/JetStreamSnapshotRestoreApiTests.cs
git commit -m "feat: add stream snapshot restore api skeleton"

Task 10: Complete Consumer API Families (List/Names/Delete)

Files:

Modify: src/NATS.Server/JetStream/Api/Handlers/ConsumerApiHandlers.cs
Modify: src/NATS.Server/JetStream/ConsumerManager.cs
Modify: src/NATS.Server/JetStream/Api/JetStreamApiResponse.cs
Test: tests/NATS.Server.Tests/JetStreamConsumerListApiTests.cs

Step 1: Write the failing test

[Fact]
public async Task Consumer_names_list_and_delete_are_supported()
{
    await using var fx = await JetStreamApiFixture.StartWithPullConsumerAsync();

    var names = await fx.RequestLocalAsync("$JS.API.CONSUMER.NAMES.ORDERS", "{}");
    names.ConsumerNames.ShouldContain("PULL");

    var del = await fx.RequestLocalAsync("$JS.API.CONSUMER.DELETE.ORDERS.PULL", "{}");
    del.Success.ShouldBeTrue();
}

Step 2: Run test to verify it fails

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamConsumerListApiTests" -v minimal
Expected: FAIL because list/delete handlers are absent.

Step 3: Write minimal implementation

public IReadOnlyList<string> ListNames(string stream)
    => _consumers.Keys.Where(k => k.Stream == stream).Select(k => k.Name).OrderBy(x => x, StringComparer.Ordinal).ToArray();

Step 4: Run test to verify it passes

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamConsumerListApiTests" -v minimal
Expected: PASS.

Step 5: Commit

git add src/NATS.Server/JetStream/Api/Handlers/ConsumerApiHandlers.cs src/NATS.Server/JetStream/ConsumerManager.cs src/NATS.Server/JetStream/Api/JetStreamApiResponse.cs tests/NATS.Server.Tests/JetStreamConsumerListApiTests.cs
git commit -m "feat: add consumer names list and delete apis"

Task 11: Implement Consumer Control APIs (Pause/Reset/Unpin)

Files:

Modify: src/NATS.Server/JetStream/Api/Handlers/ConsumerApiHandlers.cs
Modify: src/NATS.Server/JetStream/ConsumerManager.cs
Test: tests/NATS.Server.Tests/JetStreamConsumerControlApiTests.cs

Step 1: Write the failing test

[Fact]
public async Task Consumer_pause_reset_unpin_mutate_state()
{
    await using var fx = await JetStreamApiFixture.StartWithPullConsumerAsync();

    (await fx.RequestLocalAsync("$JS.API.CONSUMER.PAUSE.ORDERS.PULL", "{\"pause\":true}")).Success.ShouldBeTrue();
    (await fx.RequestLocalAsync("$JS.API.CONSUMER.RESET.ORDERS.PULL", "{}")).Success.ShouldBeTrue();
    (await fx.RequestLocalAsync("$JS.API.CONSUMER.UNPIN.ORDERS.PULL", "{}")).Success.ShouldBeTrue();
}

Step 2: Run test to verify it fails

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamConsumerControlApiTests" -v minimal
Expected: FAIL due to missing control handlers.

Step 3: Write minimal implementation

public bool Pause(string stream, string durable, bool paused)
{
    if (!TryGet(stream, durable, out var c)) return false;
    c.Paused = paused;
    return true;
}

Step 4: Run test to verify it passes

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamConsumerControlApiTests" -v minimal
Expected: PASS.

Step 5: Commit

git add src/NATS.Server/JetStream/Api/Handlers/ConsumerApiHandlers.cs src/NATS.Server/JetStream/ConsumerManager.cs tests/NATS.Server.Tests/JetStreamConsumerControlApiTests.cs
git commit -m "feat: implement consumer pause reset unpin apis"

Task 12: Implement Consumer MSG.NEXT API Contract

Files:

Modify: src/NATS.Server/JetStream/Api/Handlers/ConsumerApiHandlers.cs
Modify: src/NATS.Server/JetStream/Consumers/PullConsumerEngine.cs
Test: tests/NATS.Server.Tests/JetStreamConsumerNextApiTests.cs

Step 1: Write the failing test

[Fact]
public async Task Consumer_msg_next_respects_batch_request()
{
    await using var fx = await JetStreamApiFixture.StartWithPullConsumerAsync();
    _ = await fx.PublishAndGetAckAsync("orders.created", "1");

    var next = await fx.RequestLocalAsync("$JS.API.CONSUMER.MSG.NEXT.ORDERS.PULL", "{\"batch\":1}");
    next.PullBatch.ShouldNotBeNull();
    next.PullBatch!.Messages.Count.ShouldBe(1);
}

Step 2: Run test to verify it fails

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamConsumerNextApiTests" -v minimal
Expected: FAIL because MSG.NEXT route is unimplemented.

Step 3: Write minimal implementation

if (subject.StartsWith("$JS.API.CONSUMER.MSG.NEXT.", StringComparison.Ordinal))
    return ConsumerApiHandlers.HandleNext(subject, payload, _consumerManager, _streamManager);

Step 4: Run test to verify it passes

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamConsumerNextApiTests" -v minimal
Expected: PASS.

Step 5: Commit

git add src/NATS.Server/JetStream/Api/Handlers/ConsumerApiHandlers.cs src/NATS.Server/JetStream/Consumers/PullConsumerEngine.cs tests/NATS.Server.Tests/JetStreamConsumerNextApiTests.cs
git commit -m "feat: add consumer msg next api contract"

Task 13: Expand Stream/Consumer Policy Models and Validation

Files:

Modify: src/NATS.Server/JetStream/Models/StreamConfig.cs
Modify: src/NATS.Server/JetStream/Models/ConsumerConfig.cs
Create: src/NATS.Server/JetStream/Models/JetStreamPolicies.cs
Modify: src/NATS.Server/JetStream/Validation/JetStreamConfigValidator.cs
Test: tests/NATS.Server.Tests/JetStreamPolicyValidationTests.cs

Step 1: Write the failing test

[Fact]
public void Validator_rejects_invalid_policy_combinations()
{
    var cfg = new StreamConfig
    {
        Name = "S",
        Subjects = ["s.*"],
        Retention = RetentionPolicy.WorkQueue,
        MaxConsumers = 0,
    };

    var result = JetStreamConfigValidator.Validate(cfg);
    result.IsValid.ShouldBeFalse();
}

Step 2: Run test to verify it fails

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamPolicyValidationTests" -v minimal
Expected: FAIL due to missing policy types/validation branches.

Step 3: Write minimal implementation

public enum RetentionPolicy { Limits, Interest, WorkQueue }
public enum DiscardPolicy { Old, New }
public enum DeliverPolicy { All, Last, New, ByStartSequence, ByStartTime, LastPerSubject }
public enum ReplayPolicy { Instant, Original }

if (config.Retention == RetentionPolicy.WorkQueue && config.MaxConsumers == 0)
    return ValidationResult.Invalid("workqueue retention requires max consumers > 0");

Step 4: Run test to verify it passes

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamPolicyValidationTests" -v minimal
Expected: PASS.

Step 5: Commit

git add src/NATS.Server/JetStream/Models/StreamConfig.cs src/NATS.Server/JetStream/Models/ConsumerConfig.cs src/NATS.Server/JetStream/Models/JetStreamPolicies.cs src/NATS.Server/JetStream/Validation/JetStreamConfigValidator.cs tests/NATS.Server.Tests/JetStreamPolicyValidationTests.cs
git commit -m "feat: add jetstream policy models and validation"

Task 14: Add Expected-Header Preconditions and Dedupe Window Semantics

Files:

Modify: src/NATS.Server/JetStream/Publish/PublishPreconditions.cs
Modify: src/NATS.Server/JetStream/Publish/JetStreamPublisher.cs
Test: tests/NATS.Server.Tests/JetStreamExpectedHeaderTests.cs

Step 1: Write the failing test

[Fact]
public async Task Expected_last_sequence_mismatch_returns_error()
{
    await using var fx = await JetStreamApiFixture.StartWithStreamAsync("ORDERS", "orders.*");
    _ = await fx.PublishAndGetAckAsync("orders.created", "1");

    var ack = await fx.PublishWithExpectedLastSeqAsync("orders.created", "2", expectedLastSeq: 999);
    ack.ErrorCode.ShouldBe(10071);
}

Step 2: Run test to verify it fails

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamExpectedHeaderTests" -v minimal
Expected: FAIL because expected-sequence checks are absent.

Step 3: Write minimal implementation

public bool CheckExpectedLastSeq(ulong expectedLastSeq, ulong actualLastSeq)
    => expectedLastSeq == 0 || expectedLastSeq == actualLastSeq;

if (!_preconditions.CheckExpectedLastSeq(options.ExpectedLastSeq, state.LastSeq))
{
    ack = new PubAck { ErrorCode = 10071 };
    return true;
}

Step 4: Run test to verify it passes

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamExpectedHeaderTests" -v minimal
Expected: PASS.

Step 5: Commit

git add src/NATS.Server/JetStream/Publish/PublishPreconditions.cs src/NATS.Server/JetStream/Publish/JetStreamPublisher.cs tests/NATS.Server.Tests/JetStreamExpectedHeaderTests.cs
git commit -m "feat: add publish expected header preconditions"

Task 15: Complete Pull Consumer Semantics (`no_wait`, `expires`, `max_waiting`)

Files:

Modify: src/NATS.Server/JetStream/Consumers/PullConsumerEngine.cs
Modify: src/NATS.Server/JetStream/ConsumerManager.cs
Test: tests/NATS.Server.Tests/JetStreamPullConsumerContractTests.cs

Step 1: Write the failing test

[Fact]
public async Task Pull_fetch_no_wait_returns_immediately_when_empty()
{
    await using var fx = await JetStreamApiFixture.StartWithPullConsumerAsync();

    var batch = await fx.FetchWithNoWaitAsync("ORDERS", "PULL", batch: 1);
    batch.Messages.Count.ShouldBe(0);
    batch.TimedOut.ShouldBeFalse();
}

Step 2: Run test to verify it fails

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamPullConsumerContractTests" -v minimal
Expected: FAIL because fetch options contract is incomplete.

Step 3: Write minimal implementation

public ValueTask<PullFetchBatch> FetchAsync(StreamHandle stream, ConsumerHandle consumer, PullFetchRequest request, CancellationToken ct)
{
    if (request.NoWait && consumer.Pending.Count == 0)
        return ValueTask.FromResult(new PullFetchBatch([], timedOut: false));
    // existing fetch logic...
}

Step 4: Run test to verify it passes

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamPullConsumerContractTests" -v minimal
Expected: PASS.

Step 5: Commit

git add src/NATS.Server/JetStream/Consumers/PullConsumerEngine.cs src/NATS.Server/JetStream/ConsumerManager.cs tests/NATS.Server.Tests/JetStreamPullConsumerContractTests.cs
git commit -m "feat: implement pull consumer no-wait expires max-waiting"

Task 16: Complete Push Consumer Flow Control and Ack Variants

Files:

Modify: src/NATS.Server/JetStream/Consumers/PushConsumerEngine.cs
Modify: src/NATS.Server/JetStream/Consumers/AckProcessor.cs
Modify: src/NATS.Server/JetStream/Models/ConsumerConfig.cs
Test: tests/NATS.Server.Tests/JetStreamPushConsumerContractTests.cs

Step 1: Write the failing test

[Fact]
public async Task Ack_all_advances_floor_and_clears_pending_before_sequence()
{
    await using var fx = await JetStreamApiFixture.StartWithAckAllConsumerAsync();
    await fx.PublishManyAsync("orders.created", ["1", "2", "3"]);

    var first = await fx.FetchAsync("ORDERS", "ACKALL", 3);
    await fx.AckAllAsync("ORDERS", "ACKALL", first.Messages.Last().Sequence);

    var pending = await fx.GetPendingCountAsync("ORDERS", "ACKALL");
    pending.ShouldBe(0);
}

Step 2: Run test to verify it fails

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamPushConsumerContractTests" -v minimal
Expected: FAIL due to incomplete ack-policy handling.

Step 3: Write minimal implementation

public void AckAll(ulong sequence)
{
    foreach (var key in _pending.Keys.Where(k => k <= sequence).ToArray())
        _pending.Remove(key);
}

Step 4: Run test to verify it passes

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamPushConsumerContractTests" -v minimal
Expected: PASS.

Step 5: Commit

git add src/NATS.Server/JetStream/Consumers/PushConsumerEngine.cs src/NATS.Server/JetStream/Consumers/AckProcessor.cs src/NATS.Server/JetStream/Models/ConsumerConfig.cs tests/NATS.Server.Tests/JetStreamPushConsumerContractTests.cs
git commit -m "feat: implement push flow-control and ack policy variants"

Task 17: Extend MemStore/FileStore Indexing for Subject and Sequence Queries

Files:

Modify: src/NATS.Server/JetStream/Storage/IStreamStore.cs
Modify: src/NATS.Server/JetStream/Storage/MemStore.cs
Modify: src/NATS.Server/JetStream/Storage/FileStore.cs
Test: tests/NATS.Server.Tests/JetStreamStoreIndexTests.cs

Step 1: Write the failing test

[Fact]
public async Task Store_can_get_last_message_by_subject()
{
    var store = new MemStore();
    await store.AppendAsync("orders.created", "1"u8.ToArray(), default);
    await store.AppendAsync("orders.updated", "2"u8.ToArray(), default);
    await store.AppendAsync("orders.created", "3"u8.ToArray(), default);

    var last = await store.LoadLastBySubjectAsync("orders.created", default);
    last!.Payload.Span.SequenceEqual("3"u8).ShouldBeTrue();
}

Step 2: Run test to verify it fails

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamStoreIndexTests" -v minimal
Expected: FAIL due to missing subject index operations.

Step 3: Write minimal implementation

public ValueTask<StoredMessage?> LoadLastBySubjectAsync(string subject, CancellationToken ct)
{
    lock (_gate)
    {
        var match = _messages.Values.Where(m => m.Subject == subject).OrderByDescending(m => m.Sequence).FirstOrDefault();
        return ValueTask.FromResult(match);
    }
}

Step 4: Run test to verify it passes

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamStoreIndexTests" -v minimal
Expected: PASS.

Step 5: Commit

git add src/NATS.Server/JetStream/Storage/IStreamStore.cs src/NATS.Server/JetStream/Storage/MemStore.cs src/NATS.Server/JetStream/Storage/FileStore.cs tests/NATS.Server.Tests/JetStreamStoreIndexTests.cs
git commit -m "feat: add stream store subject and sequence indexes"

Task 18: Upgrade RAFT Safety Rules and Commit Semantics

Files:

Modify: src/NATS.Server/Raft/RaftNode.cs
Modify: src/NATS.Server/Raft/RaftReplicator.cs
Modify: src/NATS.Server/Raft/RaftLog.cs
Test: tests/NATS.Server.Tests/RaftSafetyContractTests.cs

Step 1: Write the failing test

[Fact]
public async Task Follower_rejects_stale_term_vote_and_append()
{
    var node = new RaftNode("n1");
    node.StartElection(clusterSize: 1);

    var staleVote = node.GrantVote(term: node.Term - 1);
    staleVote.Granted.ShouldBeFalse();

    await Should.ThrowAsync<InvalidOperationException>(async () =>
        await node.TryAppendFromLeaderAsync(new RaftLogEntry(1, node.Term - 1, "cmd"), default));
}

Step 2: Run test to verify it fails

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~RaftSafetyContractTests" -v minimal
Expected: FAIL because stale-term safety checks are incomplete.

Step 3: Write minimal implementation

public VoteResponse GrantVote(int term)
{
    if (term < TermState.CurrentTerm)
        return new VoteResponse { Granted = false };
    // existing logic...
}

if (entry.Term < TermState.CurrentTerm)
    throw new InvalidOperationException("stale term append rejected");

Step 4: Run test to verify it passes

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~RaftSafetyContractTests" -v minimal
Expected: PASS.

Step 5: Commit

git add src/NATS.Server/Raft/RaftNode.cs src/NATS.Server/Raft/RaftReplicator.cs src/NATS.Server/Raft/RaftLog.cs tests/NATS.Server.Tests/RaftSafetyContractTests.cs
git commit -m "feat: harden raft safety and term commit semantics"

Task 19: Implement Remaining JetStream Cluster Control APIs

Files:

Create: src/NATS.Server/JetStream/Api/Handlers/ClusterControlApiHandlers.cs
Modify: src/NATS.Server/JetStream/Cluster/JetStreamMetaGroup.cs
Modify: src/NATS.Server/JetStream/Cluster/StreamReplicaGroup.cs
Modify: src/NATS.Server/JetStream/Api/JetStreamApiRouter.cs
Test: tests/NATS.Server.Tests/JetStreamClusterControlApiTests.cs

Step 1: Write the failing test

[Fact]
public async Task Stream_leader_stepdown_and_meta_stepdown_endpoints_return_success_shape()
{
    await using var fx = await JetStreamClusterFixture.StartAsync(nodes: 3);

    var streamStepdown = await fx.RequestAsync("$JS.API.STREAM.LEADER.STEPDOWN.ORDERS", "{}");
    streamStepdown.Success.ShouldBeTrue();

    var metaStepdown = await fx.RequestAsync("$JS.API.META.LEADER.STEPDOWN", "{}");
    metaStepdown.Success.ShouldBeTrue();
}

Step 2: Run test to verify it fails

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamClusterControlApiTests" -v minimal
Expected: FAIL because control endpoints are missing.

Step 3: Write minimal implementation

public static JetStreamApiResponse HandleMetaLeaderStepdown(JetStreamMetaGroup meta)
{
    meta.StepDown();
    return JetStreamApiResponse.SuccessResponse();
}

if (subject.Equals("$JS.API.META.LEADER.STEPDOWN", StringComparison.Ordinal))
    return ClusterControlApiHandlers.HandleMetaLeaderStepdown(_metaGroup);

Step 4: Run test to verify it passes

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamClusterControlApiTests" -v minimal
Expected: PASS.

Step 5: Commit

git add src/NATS.Server/JetStream/Api/Handlers/ClusterControlApiHandlers.cs src/NATS.Server/JetStream/Cluster/JetStreamMetaGroup.cs src/NATS.Server/JetStream/Cluster/StreamReplicaGroup.cs src/NATS.Server/JetStream/Api/JetStreamApiRouter.cs tests/NATS.Server.Tests/JetStreamClusterControlApiTests.cs
git commit -m "feat: add remaining jetstream cluster control apis"

Task 20: Expand `/jsz` and JetStream `varz` Runtime Detail Fields

Files:

Modify: src/NATS.Server/Monitoring/JszHandler.cs
Modify: src/NATS.Server/Monitoring/Varz.cs
Modify: src/NATS.Server/Monitoring/VarzHandler.cs
Test: tests/NATS.Server.Tests/JetStreamMonitoringParityTests.cs

Step 1: Write the failing test

[Fact]
public async Task Jsz_and_varz_include_expanded_runtime_fields()
{
    await using var fx = await JetStreamApiFixture.StartWithPullConsumerAsync();

    var jsz = await fx.GetJszAsync();
    jsz.Streams.ShouldBeGreaterThanOrEqualTo(1);
    jsz.Consumers.ShouldBeGreaterThanOrEqualTo(1);
    jsz.ApiTotal.ShouldBeGreaterThanOrEqualTo(0);
}

Step 2: Run test to verify it fails

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamMonitoringParityTests" -v minimal
Expected: FAIL with missing runtime fields.

Step 3: Write minimal implementation

public sealed class JszResponse
{
    public ulong ApiTotal { get; set; }
    public ulong ApiErrors { get; set; }
}

ApiTotal = _server.Stats.JetStreamApiTotal,
ApiErrors = _server.Stats.JetStreamApiErrors,

Step 4: Run test to verify it passes

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamMonitoringParityTests" -v minimal
Expected: PASS.

Step 5: Commit

git add src/NATS.Server/Monitoring/JszHandler.cs src/NATS.Server/Monitoring/Varz.cs src/NATS.Server/Monitoring/VarzHandler.cs tests/NATS.Server.Tests/JetStreamMonitoringParityTests.cs
git commit -m "feat: expand jetstream jsz and varz runtime details"

Task 21: Replace Stub Integration Matrix with Real End-to-End Scenarios

Files:

Modify: tests/NATS.Server.Tests/JetStreamIntegrationMatrixTests.cs
Create: tests/NATS.Server.Tests/JetStreamIntegrationMatrix.cs

Step 1: Write the failing test

[Theory]
[InlineData("stream-msg-delete-roundtrip")]
[InlineData("consumer-msg-next-no-wait")]
[InlineData("direct-get-by-sequence")]
public async Task Integration_matrix_executes_real_server_scenarios(string scenario)
{
    var result = await JetStreamIntegrationMatrix.RunScenarioAsync(scenario);
    result.Success.ShouldBeTrue(result.Details);
}

Step 2: Run test to verify it fails

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~Integration_matrix_executes_real_server_scenarios" -v minimal
Expected: FAIL because current matrix is stubbed.

Step 3: Write minimal implementation

public static async Task<(bool Success, string Details)> RunScenarioAsync(string scenario)
{
    return scenario switch
    {
        "stream-msg-delete-roundtrip" => await ScenarioRunner.StreamMsgDeleteRoundtripAsync(),
        "consumer-msg-next-no-wait" => await ScenarioRunner.ConsumerNextNoWaitAsync(),
        "direct-get-by-sequence" => await ScenarioRunner.DirectGetBySequenceAsync(),
        _ => (false, $"unknown scenario: {scenario}"),
    };
}

Step 4: Run test to verify it passes

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStreamIntegrationMatrixTests" -v minimal
Expected: PASS.

Step 5: Commit

git add tests/NATS.Server.Tests/JetStreamIntegrationMatrixTests.cs tests/NATS.Server.Tests/JetStreamIntegrationMatrix.cs
git commit -m "test: replace jetstream integration matrix stub with real scenarios"

Task 22: Complete Dual-Gate Evidence and Update `differences.md`

Files:

Modify: docs/plans/2026-02-23-jetstream-remaining-parity-map.md
Modify: differences.md
Create: docs/plans/2026-02-23-jetstream-remaining-parity-verification.md

Step 1: Run targeted .NET JetStream/RAFT/cluster test suite

Run: dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~JetStream|FullyQualifiedName~Raft|FullyQualifiedName~Route|FullyQualifiedName~Gateway|FullyQualifiedName~Leaf" -v minimal
Expected: PASS.

Step 2: Run full .NET test suite

Run: dotnet test -v minimal
Expected: PASS.

Step 3: Update parity map with evidence links per feature row

| $JS.API.STREAM.PURGE.* | StreamApiHandlers.HandlePurge | ported | JetStreamStreamMessageApiTests.Stream_msg_get_delete_and_purge_change_state |

Step 4: Update differences.md JetStream gaps to reflect newly ported features

Run: rg -n "JETSTREAM|JetStream|\$JS.API" differences.md
Expected: entries reflect new parity status and any explicit residual deltas.

Step 5: Commit

git add docs/plans/2026-02-23-jetstream-remaining-parity-map.md docs/plans/2026-02-23-jetstream-remaining-parity-verification.md differences.md
git commit -m "docs: finalize remaining jetstream parity evidence and differences"

Dependency Order

Task 1 -> Task 2 -> Task 3 -> Task 4
Task 5 -> Task 6 -> Task 7 -> Task 8 -> Task 9
Task 10 -> Task 11 -> Task 12 -> Task 13 -> Task 14 -> Task 15 -> Task 16
Task 17 -> Task 18 -> Task 19 -> Task 20
Task 21 -> Task 22

Scope Notes for Executor

Keep implementation grounded in Go references:
- golang/nats-server/server/jetstream_api.go
- golang/nats-server/server/jetstream.go
- golang/nats-server/server/stream.go
- golang/nats-server/server/consumer.go
- golang/nats-server/server/raft.go
- golang/nats-server/server/memstore.go
- golang/nats-server/server/filestore.go
For each Go feature row added to the parity map, include exact .NET implementation path and proving test name.
Do not claim parity completion until Task 22 evidence is complete.

40 KiB Raw Blame History

JetStream Remaining Parity Implementation Plan

Task 1: Build Remaining Feature Inventory and API Matrix

Task 2: Wire $JS.API.* Request/Reply Through NATS Message Path

Task 3: Expand API Router to Full Subject Family Dispatcher

Task 4: Implement Account Info and System-Level API Responses

Task 5: Complete Stream Lifecycle APIs (Update/Delete)

Task 6: Implement Stream Names/List APIs

Task 7: Implement Stream Purge and Message Delete/Get APIs

Task 8: Implement Direct Message Get APIs

Task 9: Implement Stream Snapshot/Restore API Skeleton with Store Hooks

Task 10: Complete Consumer API Families (List/Names/Delete)

Task 11: Implement Consumer Control APIs (Pause/Reset/Unpin)

Task 12: Implement Consumer MSG.NEXT API Contract

Task 13: Expand Stream/Consumer Policy Models and Validation

Task 14: Add Expected-Header Preconditions and Dedupe Window Semantics

Task 15: Complete Pull Consumer Semantics (no_wait, expires, max_waiting)

Task 16: Complete Push Consumer Flow Control and Ack Variants

Task 17: Extend MemStore/FileStore Indexing for Subject and Sequence Queries

Task 18: Upgrade RAFT Safety Rules and Commit Semantics

Task 19: Implement Remaining JetStream Cluster Control APIs

Task 20: Expand /jsz and JetStream varz Runtime Detail Fields

Task 21: Replace Stub Integration Matrix with Real End-to-End Scenarios

Task 22: Complete Dual-Gate Evidence and Update differences.md

Dependency Order

Scope Notes for Executor

40 KiB

Raw Blame History

Task 2: Wire `$JS.API.*` Request/Reply Through NATS Message Path

Task 15: Complete Pull Consumer Semantics (`no_wait`, `expires`, `max_waiting`)

Task 20: Expand `/jsz` and JetStream `varz` Runtime Detail Fields

Task 22: Complete Dual-Gate Evidence and Update `differences.md`