docs: add remaining Go parity design — 8 tracks, ~1,418 tests
This commit is contained in:
Joseph Doherty
246
docs/plans/2026-02-24-remaining-parity-design.md
Normal file
246
docs/plans/2026-02-24-remaining-parity-design.md
Normal file
@@ -0,0 +1,246 @@
|
||||
# Remaining Go Parity: Port Missing Functionality & Tests
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:writing-plans to create the implementation plan from this design.
|
||||
|
||||
**Goal:** Port the remaining ~1,418 unmapped Go tests to .NET, building/enhancing features as needed. Mark ~224 Go-specific tests as not_applicable. Target: 90%+ Go test mapping.
|
||||
|
||||
**Baseline:** 4,886 .NET tests passing, 1,248/2,937 Go tests mapped (42.5%), 1,642 unmapped.
|
||||
|
||||
**Approach:** Feature-first by gap severity. For each gap in structuregaps.md: enhance the .NET feature, port the corresponding Go tests, update test_parity.db. Only run targeted tests during implementation; full suite at the end.
|
||||
|
||||
**Cluster testing:** Hybrid — simulated fixtures for ~270 tests, real multi-server harness for ~26 critical networking tests.
|
||||
|
||||
---
|
||||
|
||||
## Scope Classification
|
||||
|
||||
### Not Applicable (~224 tests) — mark in DB immediately
|
||||
|
||||
| Category | Count | Reason |
|
||||
|----------|------:|--------|
|
||||
| norace_1_test.go | 85 | Stress tests requiring real multi-server infrastructure |
|
||||
| norace_2_test.go | 41 | Stress tests requiring real multi-server infrastructure |
|
||||
| Performance/benchmark (`*Perf*`, `*Bench*`) | 35 | Go benchmarks, not functional tests |
|
||||
| signal_test.go | 17 | Unix signal handling, Go-specific |
|
||||
| dirstore_test.go | 17 | Go account resolver directory store |
|
||||
| split_test.go | 12 | Go test file splitting mechanism |
|
||||
| ipqueue_test.go | 9 | Go internal queue data structure |
|
||||
| certstore_windows_test.go | 4 | Windows certificate store |
|
||||
| ring_test.go | 2 | Go ring buffer utility |
|
||||
| service_windows_test.go | 1 | Windows service manager |
|
||||
| service_test.go | 1 | Go-specific service test |
|
||||
|
||||
### Portable (~1,418 tests) — organized into 8 tracks
|
||||
|
||||
---
|
||||
|
||||
## Track 1: Storage Engine (167 tests)
|
||||
|
||||
**Gaps addressed:** Gap 1 (FileStore Block Management — CRITICAL)
|
||||
|
||||
**Go test files:**
|
||||
- `filestore_test.go`: 123 unmapped
|
||||
- `memstore_test.go`: 28 unmapped
|
||||
- `store_test.go`: 16 unmapped
|
||||
|
||||
**Current .NET state:** FileStore.cs (1,587 lines), MsgBlock.cs (605 lines), MemStore.cs (160 lines), AeadEncryptor.cs (165 lines), S2Codec.cs (111 lines)
|
||||
|
||||
**Feature work needed:**
|
||||
1. **Crash recovery**: Rebuild block indexes from raw data when index corrupt/missing. Go: `recoverFullState()`, `rebuildState()`.
|
||||
2. **Consumer state serialization**: Encode/decode pending acks, redelivered sets, pending-below-ack-floor into consumer store. Go: `writeConsumerState()`, `readConsumerState()`.
|
||||
3. **Write failure resilience**: Handle disk-full, permission errors gracefully during `StoreMsg()`. Partial write recovery.
|
||||
4. **Block compaction**: Remove tombstones, reclaim space from deleted messages within blocks.
|
||||
5. **MemStore enhancements**: TTL edge cases, limits enforcement, state recovery after restart.
|
||||
|
||||
**Dependencies:** None — foundational track.
|
||||
|
||||
---
|
||||
|
||||
## Track 2: JetStream Core API (220 tests)
|
||||
|
||||
**Gaps addressed:** Gap 7 (JetStream API — HIGH), Gap 3 (Mirrors/Sources — HIGH)
|
||||
|
||||
**Go test files:**
|
||||
- `jetstream_test.go`: 173 unmapped
|
||||
- `jetstream_batching_test.go`: 29 unmapped
|
||||
- `jetstream_versioning_test.go`: 18 unmapped
|
||||
|
||||
**Current .NET state:** StreamApiHandlers.cs (404 lines), JetStreamApiRouter.cs (203 lines), StreamManager.cs (529 lines), MirrorCoordinator.cs (364 lines), SourceCoordinator.cs (470 lines)
|
||||
|
||||
**Feature work needed:**
|
||||
1. **Dedup window**: Track `Nats-Msg-Id` headers per stream with configurable TTL. Go: `checkMsgId()`, `storeMsgId()`.
|
||||
2. **Purge with filter**: `$JS.API.STREAM.PURGE` with subject filter and `keep` parameter.
|
||||
3. **Snapshot/restore**: API endpoints for stream backup and restore.
|
||||
4. **Tiered limits**: Per-account JetStream storage limits by tier (R1, R3).
|
||||
5. **Batch publish**: Handle multi-message publish with atomic success/failure semantics.
|
||||
6. **Version negotiation**: API version header support for backward compatibility.
|
||||
7. **Interest retention edge cases**: Correct wildcard/filtered consumer interaction with interest retention.
|
||||
|
||||
**Dependencies:** None — independent of Track 1.
|
||||
|
||||
---
|
||||
|
||||
## Track 3: JetStream Cluster (296 tests)
|
||||
|
||||
**Gaps addressed:** Gap 2 (JetStream Cluster Coordination — CRITICAL)
|
||||
|
||||
**Go test files:**
|
||||
- `jetstream_cluster_1_test.go`: 73 unmapped
|
||||
- `jetstream_cluster_2_test.go`: 92 unmapped
|
||||
- `jetstream_cluster_3_test.go`: 64 unmapped
|
||||
- `jetstream_cluster_4_test.go`: 61 unmapped
|
||||
- `jetstream_cluster_long_test.go`: 6 unmapped
|
||||
|
||||
**Current .NET state:** JetStreamMetaGroup.cs (454 lines), StreamReplicaGroup.cs (300 lines), PlacementEngine.cs (80 lines), JetStreamClusterFixture (existing test infrastructure)
|
||||
|
||||
**Feature work needed:**
|
||||
1. **Simulated fixture enhancements**: Multi-node behavior simulation (3-node, not just leader), partition simulation (isolate/heal nodes), inflight request dedup.
|
||||
2. **Multi-server harness** (new ~26 tests): `MultiServerFixture` that starts real NATS.Server.Host processes, configures routes via temp config files, verifies route establishment, tears down on dispose.
|
||||
3. **Meta-group enhancements**: Peer removal simulation, stream/consumer assignment version tracking, step-down cascading behavior.
|
||||
|
||||
**Testing approach:**
|
||||
- ~270 tests: Simulated fixtures (extend JetStreamClusterFixture)
|
||||
- ~26 tests: Real multi-server harness (leader election, partition behavior, route failover)
|
||||
|
||||
**Dependencies:** Track 1 (storage must be solid for cluster storage tests).
|
||||
|
||||
---
|
||||
|
||||
## Track 4: Consumer Engines (96 tests)
|
||||
|
||||
**Gaps addressed:** Gap 4 (Consumer Delivery Engines — HIGH)
|
||||
|
||||
**Go test files:**
|
||||
- `jetstream_consumer_test.go`: 96 unmapped
|
||||
|
||||
**Current .NET state:** PushConsumerEngine.cs (264 lines), PullConsumerEngine.cs (300 lines), AckProcessor.cs (225 lines), PriorityGroupManager.cs (102 lines), RedeliveryTracker.cs (92 lines)
|
||||
|
||||
**Feature work needed:**
|
||||
1. **Backoff schedules**: Configurable per-message redelivery delay arrays in RedeliveryTracker. Go: `getNextRetryDelay()`.
|
||||
2. **Pause/resume**: Consumer state machine with `Pause(until)`, `Resume()`, advisory publication.
|
||||
3. **Idle heartbeats**: Timer-based heartbeat generation when no messages delivered within configured interval.
|
||||
4. **Max deliveries**: Enforcement with configurable dead-letter to `$JS.EVENT.ADVISORY.CONSUMER.MAX_DELIVERIES`.
|
||||
5. **Replay rate**: Deliver at original publish timestamps for `replay=original` mode.
|
||||
6. **Filter subject skip tracking**: Skip non-matching messages without counting against pending limits.
|
||||
|
||||
**Dependencies:** None — uses existing storage interface.
|
||||
|
||||
---
|
||||
|
||||
## Track 5: Auth/JWT/Accounts (105 tests)
|
||||
|
||||
**Gaps addressed:** Gap 9 (Account Management — MEDIUM)
|
||||
|
||||
**Go test files:**
|
||||
- `jwt_test.go`: 61 unmapped
|
||||
- `auth_callout_test.go`: 30 unmapped
|
||||
- `accounts_test.go`: 14 unmapped
|
||||
|
||||
**Current .NET state:** JwtAuthenticator.cs (180 lines), AccountClaims.cs (107 lines), AuthService.cs (172 lines), Account.cs (189 lines), various auth files (~1,266 total)
|
||||
|
||||
**Feature work needed:**
|
||||
1. **JWT claims depth**: Full operator→account→user trust chain validation. Signing key hierarchy.
|
||||
2. **Auth callout**: Wire protocol for delegating auth to external service via NATS request/reply.
|
||||
3. **Account imports/exports**: Runtime enforcement of service/stream exports with authorization checks and cycle detection.
|
||||
4. **Account resource limits**: Per-account JetStream limits (storage, streams, consumers).
|
||||
|
||||
**Dependencies:** None.
|
||||
|
||||
---
|
||||
|
||||
## Track 6: Networking (140 tests + 47 super-cluster)
|
||||
|
||||
**Gaps addressed:** Gap 11 (Gateway — MEDIUM), Gap 12 (Leaf Node — MEDIUM), Gap 13 (Routes — MEDIUM)
|
||||
|
||||
**Go test files:**
|
||||
- `gateway_test.go`: 48 unmapped
|
||||
- `leafnode_test.go`: 55 unmapped
|
||||
- `routes_test.go`: 37 unmapped
|
||||
- `leafnode_proxy_test.go`: 8 unmapped
|
||||
- `jetstream_leafnode_test.go`: 12 unmapped
|
||||
- `jetstream_super_cluster_test.go`: 47 unmapped
|
||||
|
||||
**Current .NET state:** GatewayManager.cs (225 lines), GatewayConnection.cs (242 lines), LeafNodeManager.cs (213 lines), LeafConnection.cs (242 lines), RouteManager.cs (269 lines), RouteConnection.cs (289 lines)
|
||||
|
||||
**Feature work needed:**
|
||||
1. **Gateway interest-only mode**: Switch from flooding to interest-based forwarding after subscription snapshot.
|
||||
2. **Route pooling**: Multiple connections per peer, round-robin distribution.
|
||||
3. **Leaf solicited connections**: Outbound leaf connections with retry and backoff.
|
||||
4. **Route/leaf compression**: Wire S2Codec into route and leaf connection paths.
|
||||
5. **Super-cluster simulation**: Multi-cluster topology via gateway simulation in test fixtures.
|
||||
6. **Leaf JetStream domain**: Forward JetStream domain headers through leaf connections.
|
||||
|
||||
**Dependencies:** None.
|
||||
|
||||
---
|
||||
|
||||
## Track 7: Config/Reload/Monitoring (132 tests)
|
||||
|
||||
**Gaps addressed:** Gap 14 (Configuration — MEDIUM), Gap 10 (Monitoring — MEDIUM)
|
||||
|
||||
**Go test files:**
|
||||
- `opts_test.go`: 49 unmapped
|
||||
- `reload_test.go`: 38 unmapped
|
||||
- `monitor_test.go`: 45 unmapped
|
||||
|
||||
**Current .NET state:** ConfigProcessor.cs (1,023 lines), NatsConfLexer.cs (1,503 lines), ConfigReloader.cs (526 lines), Monitoring/ (1,698 lines)
|
||||
|
||||
**Feature work needed:**
|
||||
1. **Auth change propagation**: On reload, disconnect clients with revoked credentials.
|
||||
2. **TLS cert reload**: Reload certificates without restart.
|
||||
3. **CLI flag coverage**: Parse remaining Go CLI flags.
|
||||
4. **Monitoring completeness**: Add missing sort options, account filtering to `/connz`. Closed connection ring buffer.
|
||||
5. **Event payload completeness**: Full JSON payloads for connect/disconnect/auth-error events.
|
||||
|
||||
**Dependencies:** None.
|
||||
|
||||
---
|
||||
|
||||
## Track 8: Client/Protocol/Misc (112 tests)
|
||||
|
||||
**Gaps addressed:** Gap 5 (Client Protocol — HIGH), misc
|
||||
|
||||
**Go test files:**
|
||||
- `client_test.go`: 22 unmapped
|
||||
- `server_test.go`: 28 unmapped
|
||||
- `sublist_test.go`: 25 unmapped
|
||||
- `msgtrace_test.go`: 22 unmapped
|
||||
- `client_proxyproto_test.go`: 23 unmapped
|
||||
- Smaller: parser (5), closed_conns (6), log (4), errors (2), config_check (2), auth_test (2), jetstream_errors (4), jetstream_jwt (18), jetstream_tpm (5), trust (3), subject_transform (3), nkey (1), ping (1), rate_counter (1), util (6), log (4), mqtt_ex (2), closed_conns (6)
|
||||
|
||||
**Current .NET state:** NatsClient.cs (924 lines), NatsParser.cs (495 lines), SubList.cs (984 lines), MessageTraceContext.cs (686 lines)
|
||||
|
||||
**Feature work needed:**
|
||||
1. **PROXY protocol**: Parse v1/v2 header for client source IP behind load balancers.
|
||||
2. **Message trace propagation**: Full trace through publish→match→route→deliver pipeline.
|
||||
3. **Slow consumer detection**: Pending bytes/messages threshold with eviction advisory.
|
||||
4. **Adaptive read buffer**: Dynamic resize from 512→65KB based on throughput.
|
||||
5. **SubList stress tests**: Concurrent insert/match/remove under load.
|
||||
|
||||
**Dependencies:** None.
|
||||
|
||||
---
|
||||
|
||||
## Execution Strategy
|
||||
|
||||
### Parallelism
|
||||
- **Independent tracks** (can start immediately): 1, 2, 4, 5, 6, 7, 8
|
||||
- **Dependent track**: 3 (waits for Track 1 completion)
|
||||
- Within each track, subagents work sequentially on feature+test batches
|
||||
|
||||
### DB Update Pattern
|
||||
After each batch of tests:
|
||||
```sql
|
||||
UPDATE go_tests SET status='mapped', dotnet_test='DotNetTestName', dotnet_file='TestFile.cs'
|
||||
WHERE go_test='GoTestName' AND go_file='go_file.go';
|
||||
```
|
||||
|
||||
### Test Execution
|
||||
- **During implementation**: `dotnet test --filter "FullyQualifiedName~ClassName"` — targeted only
|
||||
- **End of all tracks**: `dotnet test` — full suite verification
|
||||
|
||||
### Expected Outcome
|
||||
- ~224 tests marked `not_applicable`
|
||||
- ~1,418 tests mapped with passing .NET equivalents
|
||||
- Final mapping: ~2,666/2,937 (90.8%)
|
||||
- .NET test count: ~6,300+ (4,886 current + ~1,418 new)
|
||||
Reference in New Issue
Block a user