Files
natsdotnet/docs/plans/2026-03-12-e2e-extended-design.md
Joseph Doherty c30e67a69d Fix E2E test gaps and add comprehensive E2E + parity test suites
- Fix pull consumer fetch: send original stream subject in HMSG (not inbox)
  so NATS client distinguishes data messages from control messages
- Fix MaxAge expiry: add background timer in StreamManager for periodic pruning
- Fix JetStream wire format: Go-compatible anonymous objects with string enums,
  proper offset-based pagination for stream/consumer list APIs
- Add 42 E2E black-box tests (core messaging, auth, TLS, accounts, JetStream)
- Add ~1000 parity tests across all subsystems (gaps closure)
- Update gap inventory docs to reflect implementation status
2026-03-12 14:09:23 -04:00

6.9 KiB

NATS.E2E.Tests Extended Coverage — Design

Date: 2026-03-12 Status: Approved

Overview

Extend the existing NATS.E2E.Tests project with phased test coverage that progressively exercises more complex NATS server functionality. All tests are black-box: the server runs as a child process, tests connect via NATS.Client.Core (and NATS.Client.JetStream for Phase 5).

Infrastructure Extensions

NatsServerProcess Changes

  • Add optional string[]? extraArgs to constructor, appended after -p <port>
  • Add WithConfigFile(string content) static factory — writes content to a temp file, passes -c <path>, cleans up on dispose
  • Add MonitorPort property — allocates a second free port when monitoring is needed, passes -m <port>

New Fixtures (Infrastructure/)

Each fixture gets its own [CollectionDefinition]:

Fixture Collection Config
NatsServerFixture (existing) E2E Default, no auth/TLS/JS
AuthServerFixture E2E-Auth Config file with users, tokens, NKeys, permissions
MonitorServerFixture E2E-Monitor -m <port> for HTTP monitoring
TlsServerFixture E2E-TLS Self-signed certs generated at startup, --tlscert/--tlskey/--tlscacert
AccountServerFixture E2E-Accounts Config with two isolated accounts
JetStreamServerFixture E2E-JetStream Config with jetstream { store_dir: <tmpdir> }

Shared Helper

E2ETestHelper static class:

  • CreateClient(int port) — returns NatsConnection
  • Timeout(int seconds = 10) — returns CancellationToken

NuGet Additions

  • NATS.NKeys — already in Directory.Packages.props, add to E2E csproj
  • NATS.Client.JetStream — add to Directory.Packages.props and E2E csproj

Phase 1: Core Messaging (11 tests)

File: CoreMessagingTests.cs[Collection("E2E")]

Test Verifies
WildcardStar_MatchesSingleToken Sub foo.*, pub foo.bar → received
WildcardGreaterThan_MatchesMultipleTokens Sub foo.>, pub foo.bar.baz → received
WildcardStar_DoesNotMatchMultipleTokens Sub foo.*, pub foo.bar.baz → no message
QueueGroup_LoadBalances 3 queue subs, 30 msgs → distributed across all 3
QueueGroup_MixedWithPlainSub 1 plain + 2 queue subs → plain gets all, 1 queue gets each
Unsub_StopsDelivery Sub, unsub, pub → no message
Unsub_WithMaxMessages Auto-unsub after 3, pub 5 → only 3 received
FanOut_MultipleSubscribers 3 subs, 1 pub → all 3 receive
EchoOff_PublisherDoesNotReceiveSelf echo: false, self-pub → no echo
VerboseMode_OkResponses Raw socket, verbose: true+OK after SUB
NoResponders_Returns503 no_responders: true, request with no subs → 503

Phase 2: Auth & Permissions (12 tests)

File: AuthTests.cs[Collection("E2E-Auth")]

Config includes: user/pass pair, token, NKey public key, permission-restricted users, max_subs: 5 user.

Test Verifies
UsernamePassword_ValidCredentials_Connects Correct user/pass → connected
UsernamePassword_InvalidPassword_Rejected Wrong pass → rejected
UsernamePassword_NoCredentials_Rejected No creds to auth server → rejected
TokenAuth_ValidToken_Connects Correct token → connected
TokenAuth_InvalidToken_Rejected Wrong token → rejected
NKeyAuth_ValidSignature_Connects Valid NKey sig → connected
NKeyAuth_InvalidSignature_Rejected Wrong NKey sig → rejected
Permission_PublishAllowed_Succeeds Pub to allowed subject → delivered
Permission_PublishDenied_NoDelivery Pub to denied subject → not delivered
Permission_SubscribeDenied_Rejected Sub to denied subject → rejected
MaxSubscriptions_ExceedsLimit_Rejected 6th sub on max_subs: 5 user → rejected
MaxPayload_ExceedsLimit_Disconnected Oversized message → disconnected

Phase 3: Monitoring & Config (7 tests)

File: MonitoringTests.cs[Collection("E2E-Monitor")]

Fixture exposes MonitorPort and HttpClient MonitorClient.

Test Verifies
Healthz_ReturnsOk GET /healthz → 200, {"status":"ok"}
Varz_ReturnsServerInfo GET /varz → JSON with server_id, version, port
Varz_ReflectsMessageCounts Publish msgs, GET /varzin_msgs > 0
Connz_ListsActiveConnections 2 clients, GET /connznum_connections: 2
Connz_SortByParameter GET /connz?sort=bytes_to → sorted
Connz_LimitAndOffset 5 clients, GET /connz?limit=2&offset=1 → 2 entries
Subz_ReturnsSubscriptionStats Subs active, GET /subz → count > 0

Phase 4: TLS & Account Isolation (6 tests)

File: TlsTests.cs[Collection("E2E-TLS")]

Fixture generates self-signed CA + server cert + client cert using System.Security.Cryptography at startup.

Test Verifies
Tls_ClientConnectsSecurely TLS connect + ping succeeds
Tls_PlainTextConnection_Rejected Non-TLS connect → fails
Tls_PubSub_WorksOverEncryptedConnection Full pub/sub over TLS

File: AccountIsolationTests.cs[Collection("E2E-Accounts")]

Config with ACCT_A and ACCT_B, each with its own user.

Test Verifies
Accounts_SameAccount_MessageDelivered Two ACCT_A clients → pub/sub works
Accounts_CrossAccount_MessageNotDelivered ACCT_A pub, ACCT_B sub → no message
Accounts_EachAccountHasOwnNamespace Both accounts sub foo.bar independently

Phase 5: JetStream (10 tests)

File: JetStreamTests.cs[Collection("E2E-JetStream")]

Fixture enables JetStream with temp store_dir.

Test Verifies
Stream_CreateAndInfo Create stream, verify info matches config
Stream_ListAndNames Create 3 streams, list/names returns all
Stream_Delete Create, delete, verify gone
Stream_PublishAndGet Publish msgs, get by sequence
Stream_Purge Publish, purge, verify count = 0
Consumer_CreatePullAndConsume Pull consumer, publish 5, pull → receive 5
Consumer_AckExplicit Explicit ack, verify no redelivery
Consumer_ListAndDelete Create consumers, list, delete, verify
Retention_LimitsMaxMessages max_msgs: 10, publish 15 → stream has 10
Retention_MaxAge Short max_age, verify msgs expire

Test Count Summary

Phase Tests New Files
1 — Core Messaging 11 CoreMessagingTests.cs
2 — Auth & Permissions 12 AuthTests.cs, AuthServerFixture.cs
3 — Monitoring 7 MonitoringTests.cs, MonitorServerFixture.cs
4 — TLS & Accounts 6 TlsTests.cs, TlsServerFixture.cs, AccountIsolationTests.cs, AccountServerFixture.cs
5 — JetStream 10 JetStreamTests.cs, JetStreamServerFixture.cs
Total 46

Plus infrastructure changes: NatsServerProcess.cs (edit), E2ETestHelper.cs (new).