Files
natsdotnet/docs/plans/2026-02-24-remaining-parity-design.md

11 KiB

Remaining Go Parity: Port Missing Functionality & Tests

For Claude: REQUIRED SUB-SKILL: Use superpowers-extended-cc:writing-plans to create the implementation plan from this design.

Goal: Port the remaining ~1,418 unmapped Go tests to .NET, building/enhancing features as needed. Mark ~224 Go-specific tests as not_applicable. Target: 90%+ Go test mapping.

Baseline: 4,886 .NET tests passing, 1,248/2,937 Go tests mapped (42.5%), 1,642 unmapped.

Approach: Feature-first by gap severity. For each gap in structuregaps.md: enhance the .NET feature, port the corresponding Go tests, update test_parity.db. Only run targeted tests during implementation; full suite at the end.

Cluster testing: Hybrid — simulated fixtures for ~270 tests, real multi-server harness for ~26 critical networking tests.


Scope Classification

Not Applicable (~224 tests) — mark in DB immediately

Category Count Reason
norace_1_test.go 85 Stress tests requiring real multi-server infrastructure
norace_2_test.go 41 Stress tests requiring real multi-server infrastructure
Performance/benchmark (*Perf*, *Bench*) 35 Go benchmarks, not functional tests
signal_test.go 17 Unix signal handling, Go-specific
dirstore_test.go 17 Go account resolver directory store
split_test.go 12 Go test file splitting mechanism
ipqueue_test.go 9 Go internal queue data structure
certstore_windows_test.go 4 Windows certificate store
ring_test.go 2 Go ring buffer utility
service_windows_test.go 1 Windows service manager
service_test.go 1 Go-specific service test

Portable (~1,418 tests) — organized into 8 tracks


Track 1: Storage Engine (167 tests)

Gaps addressed: Gap 1 (FileStore Block Management — CRITICAL)

Go test files:

  • filestore_test.go: 123 unmapped
  • memstore_test.go: 28 unmapped
  • store_test.go: 16 unmapped

Current .NET state: FileStore.cs (1,587 lines), MsgBlock.cs (605 lines), MemStore.cs (160 lines), AeadEncryptor.cs (165 lines), S2Codec.cs (111 lines)

Feature work needed:

  1. Crash recovery: Rebuild block indexes from raw data when index corrupt/missing. Go: recoverFullState(), rebuildState().
  2. Consumer state serialization: Encode/decode pending acks, redelivered sets, pending-below-ack-floor into consumer store. Go: writeConsumerState(), readConsumerState().
  3. Write failure resilience: Handle disk-full, permission errors gracefully during StoreMsg(). Partial write recovery.
  4. Block compaction: Remove tombstones, reclaim space from deleted messages within blocks.
  5. MemStore enhancements: TTL edge cases, limits enforcement, state recovery after restart.

Dependencies: None — foundational track.


Track 2: JetStream Core API (220 tests)

Gaps addressed: Gap 7 (JetStream API — HIGH), Gap 3 (Mirrors/Sources — HIGH)

Go test files:

  • jetstream_test.go: 173 unmapped
  • jetstream_batching_test.go: 29 unmapped
  • jetstream_versioning_test.go: 18 unmapped

Current .NET state: StreamApiHandlers.cs (404 lines), JetStreamApiRouter.cs (203 lines), StreamManager.cs (529 lines), MirrorCoordinator.cs (364 lines), SourceCoordinator.cs (470 lines)

Feature work needed:

  1. Dedup window: Track Nats-Msg-Id headers per stream with configurable TTL. Go: checkMsgId(), storeMsgId().
  2. Purge with filter: $JS.API.STREAM.PURGE with subject filter and keep parameter.
  3. Snapshot/restore: API endpoints for stream backup and restore.
  4. Tiered limits: Per-account JetStream storage limits by tier (R1, R3).
  5. Batch publish: Handle multi-message publish with atomic success/failure semantics.
  6. Version negotiation: API version header support for backward compatibility.
  7. Interest retention edge cases: Correct wildcard/filtered consumer interaction with interest retention.

Dependencies: None — independent of Track 1.


Track 3: JetStream Cluster (296 tests)

Gaps addressed: Gap 2 (JetStream Cluster Coordination — CRITICAL)

Go test files:

  • jetstream_cluster_1_test.go: 73 unmapped
  • jetstream_cluster_2_test.go: 92 unmapped
  • jetstream_cluster_3_test.go: 64 unmapped
  • jetstream_cluster_4_test.go: 61 unmapped
  • jetstream_cluster_long_test.go: 6 unmapped

Current .NET state: JetStreamMetaGroup.cs (454 lines), StreamReplicaGroup.cs (300 lines), PlacementEngine.cs (80 lines), JetStreamClusterFixture (existing test infrastructure)

Feature work needed:

  1. Simulated fixture enhancements: Multi-node behavior simulation (3-node, not just leader), partition simulation (isolate/heal nodes), inflight request dedup.
  2. Multi-server harness (new ~26 tests): MultiServerFixture that starts real NATS.Server.Host processes, configures routes via temp config files, verifies route establishment, tears down on dispose.
  3. Meta-group enhancements: Peer removal simulation, stream/consumer assignment version tracking, step-down cascading behavior.

Testing approach:

  • ~270 tests: Simulated fixtures (extend JetStreamClusterFixture)
  • ~26 tests: Real multi-server harness (leader election, partition behavior, route failover)

Dependencies: Track 1 (storage must be solid for cluster storage tests).


Track 4: Consumer Engines (96 tests)

Gaps addressed: Gap 4 (Consumer Delivery Engines — HIGH)

Go test files:

  • jetstream_consumer_test.go: 96 unmapped

Current .NET state: PushConsumerEngine.cs (264 lines), PullConsumerEngine.cs (300 lines), AckProcessor.cs (225 lines), PriorityGroupManager.cs (102 lines), RedeliveryTracker.cs (92 lines)

Feature work needed:

  1. Backoff schedules: Configurable per-message redelivery delay arrays in RedeliveryTracker. Go: getNextRetryDelay().
  2. Pause/resume: Consumer state machine with Pause(until), Resume(), advisory publication.
  3. Idle heartbeats: Timer-based heartbeat generation when no messages delivered within configured interval.
  4. Max deliveries: Enforcement with configurable dead-letter to $JS.EVENT.ADVISORY.CONSUMER.MAX_DELIVERIES.
  5. Replay rate: Deliver at original publish timestamps for replay=original mode.
  6. Filter subject skip tracking: Skip non-matching messages without counting against pending limits.

Dependencies: None — uses existing storage interface.


Track 5: Auth/JWT/Accounts (105 tests)

Gaps addressed: Gap 9 (Account Management — MEDIUM)

Go test files:

  • jwt_test.go: 61 unmapped
  • auth_callout_test.go: 30 unmapped
  • accounts_test.go: 14 unmapped

Current .NET state: JwtAuthenticator.cs (180 lines), AccountClaims.cs (107 lines), AuthService.cs (172 lines), Account.cs (189 lines), various auth files (~1,266 total)

Feature work needed:

  1. JWT claims depth: Full operator→account→user trust chain validation. Signing key hierarchy.
  2. Auth callout: Wire protocol for delegating auth to external service via NATS request/reply.
  3. Account imports/exports: Runtime enforcement of service/stream exports with authorization checks and cycle detection.
  4. Account resource limits: Per-account JetStream limits (storage, streams, consumers).

Dependencies: None.


Track 6: Networking (140 tests + 47 super-cluster)

Gaps addressed: Gap 11 (Gateway — MEDIUM), Gap 12 (Leaf Node — MEDIUM), Gap 13 (Routes — MEDIUM)

Go test files:

  • gateway_test.go: 48 unmapped
  • leafnode_test.go: 55 unmapped
  • routes_test.go: 37 unmapped
  • leafnode_proxy_test.go: 8 unmapped
  • jetstream_leafnode_test.go: 12 unmapped
  • jetstream_super_cluster_test.go: 47 unmapped

Current .NET state: GatewayManager.cs (225 lines), GatewayConnection.cs (242 lines), LeafNodeManager.cs (213 lines), LeafConnection.cs (242 lines), RouteManager.cs (269 lines), RouteConnection.cs (289 lines)

Feature work needed:

  1. Gateway interest-only mode: Switch from flooding to interest-based forwarding after subscription snapshot.
  2. Route pooling: Multiple connections per peer, round-robin distribution.
  3. Leaf solicited connections: Outbound leaf connections with retry and backoff.
  4. Route/leaf compression: Wire S2Codec into route and leaf connection paths.
  5. Super-cluster simulation: Multi-cluster topology via gateway simulation in test fixtures.
  6. Leaf JetStream domain: Forward JetStream domain headers through leaf connections.

Dependencies: None.


Track 7: Config/Reload/Monitoring (132 tests)

Gaps addressed: Gap 14 (Configuration — MEDIUM), Gap 10 (Monitoring — MEDIUM)

Go test files:

  • opts_test.go: 49 unmapped
  • reload_test.go: 38 unmapped
  • monitor_test.go: 45 unmapped

Current .NET state: ConfigProcessor.cs (1,023 lines), NatsConfLexer.cs (1,503 lines), ConfigReloader.cs (526 lines), Monitoring/ (1,698 lines)

Feature work needed:

  1. Auth change propagation: On reload, disconnect clients with revoked credentials.
  2. TLS cert reload: Reload certificates without restart.
  3. CLI flag coverage: Parse remaining Go CLI flags.
  4. Monitoring completeness: Add missing sort options, account filtering to /connz. Closed connection ring buffer.
  5. Event payload completeness: Full JSON payloads for connect/disconnect/auth-error events.

Dependencies: None.


Track 8: Client/Protocol/Misc (112 tests)

Gaps addressed: Gap 5 (Client Protocol — HIGH), misc

Go test files:

  • client_test.go: 22 unmapped
  • server_test.go: 28 unmapped
  • sublist_test.go: 25 unmapped
  • msgtrace_test.go: 22 unmapped
  • client_proxyproto_test.go: 23 unmapped
  • Smaller: parser (5), closed_conns (6), log (4), errors (2), config_check (2), auth_test (2), jetstream_errors (4), jetstream_jwt (18), jetstream_tpm (5), trust (3), subject_transform (3), nkey (1), ping (1), rate_counter (1), util (6), log (4), mqtt_ex (2), closed_conns (6)

Current .NET state: NatsClient.cs (924 lines), NatsParser.cs (495 lines), SubList.cs (984 lines), MessageTraceContext.cs (686 lines)

Feature work needed:

  1. PROXY protocol: Parse v1/v2 header for client source IP behind load balancers.
  2. Message trace propagation: Full trace through publish→match→route→deliver pipeline.
  3. Slow consumer detection: Pending bytes/messages threshold with eviction advisory.
  4. Adaptive read buffer: Dynamic resize from 512→65KB based on throughput.
  5. SubList stress tests: Concurrent insert/match/remove under load.

Dependencies: None.


Execution Strategy

Parallelism

  • Independent tracks (can start immediately): 1, 2, 4, 5, 6, 7, 8
  • Dependent track: 3 (waits for Track 1 completion)
  • Within each track, subagents work sequentially on feature+test batches

DB Update Pattern

After each batch of tests:

UPDATE go_tests SET status='mapped', dotnet_test='DotNetTestName', dotnet_file='TestFile.cs'
  WHERE go_test='GoTestName' AND go_file='go_file.go';

Test Execution

  • During implementation: dotnet test --filter "FullyQualifiedName~ClassName" — targeted only
  • End of all tracks: dotnet test — full suite verification

Expected Outcome

  • ~224 tests marked not_applicable
  • ~1,418 tests mapped with passing .NET equivalents
  • Final mapping: ~2,666/2,937 (90.8%)
  • .NET test count: ~6,300+ (4,886 current + ~1,418 new)