From 4ab4f578e3cb8e7fe935c6d388b35dd3461474ee Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Tue, 24 Feb 2026 19:10:40 -0500 Subject: [PATCH] =?UTF-8?q?docs:=20add=20remaining=20Go=20parity=20design?= =?UTF-8?q?=20=E2=80=94=208=20tracks,=20~1,418=20tests?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../2026-02-24-remaining-parity-design.md | 246 ++++++++++++++++++ 1 file changed, 246 insertions(+) create mode 100644 docs/plans/2026-02-24-remaining-parity-design.md diff --git a/docs/plans/2026-02-24-remaining-parity-design.md b/docs/plans/2026-02-24-remaining-parity-design.md new file mode 100644 index 0000000..5ed8968 --- /dev/null +++ b/docs/plans/2026-02-24-remaining-parity-design.md @@ -0,0 +1,246 @@ +# Remaining Go Parity: Port Missing Functionality & Tests + +> **For Claude:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:writing-plans to create the implementation plan from this design. + +**Goal:** Port the remaining ~1,418 unmapped Go tests to .NET, building/enhancing features as needed. Mark ~224 Go-specific tests as not_applicable. Target: 90%+ Go test mapping. + +**Baseline:** 4,886 .NET tests passing, 1,248/2,937 Go tests mapped (42.5%), 1,642 unmapped. + +**Approach:** Feature-first by gap severity. For each gap in structuregaps.md: enhance the .NET feature, port the corresponding Go tests, update test_parity.db. Only run targeted tests during implementation; full suite at the end. + +**Cluster testing:** Hybrid — simulated fixtures for ~270 tests, real multi-server harness for ~26 critical networking tests. + +--- + +## Scope Classification + +### Not Applicable (~224 tests) — mark in DB immediately + +| Category | Count | Reason | +|----------|------:|--------| +| norace_1_test.go | 85 | Stress tests requiring real multi-server infrastructure | +| norace_2_test.go | 41 | Stress tests requiring real multi-server infrastructure | +| Performance/benchmark (`*Perf*`, `*Bench*`) | 35 | Go benchmarks, not functional tests | +| signal_test.go | 17 | Unix signal handling, Go-specific | +| dirstore_test.go | 17 | Go account resolver directory store | +| split_test.go | 12 | Go test file splitting mechanism | +| ipqueue_test.go | 9 | Go internal queue data structure | +| certstore_windows_test.go | 4 | Windows certificate store | +| ring_test.go | 2 | Go ring buffer utility | +| service_windows_test.go | 1 | Windows service manager | +| service_test.go | 1 | Go-specific service test | + +### Portable (~1,418 tests) — organized into 8 tracks + +--- + +## Track 1: Storage Engine (167 tests) + +**Gaps addressed:** Gap 1 (FileStore Block Management — CRITICAL) + +**Go test files:** +- `filestore_test.go`: 123 unmapped +- `memstore_test.go`: 28 unmapped +- `store_test.go`: 16 unmapped + +**Current .NET state:** FileStore.cs (1,587 lines), MsgBlock.cs (605 lines), MemStore.cs (160 lines), AeadEncryptor.cs (165 lines), S2Codec.cs (111 lines) + +**Feature work needed:** +1. **Crash recovery**: Rebuild block indexes from raw data when index corrupt/missing. Go: `recoverFullState()`, `rebuildState()`. +2. **Consumer state serialization**: Encode/decode pending acks, redelivered sets, pending-below-ack-floor into consumer store. Go: `writeConsumerState()`, `readConsumerState()`. +3. **Write failure resilience**: Handle disk-full, permission errors gracefully during `StoreMsg()`. Partial write recovery. +4. **Block compaction**: Remove tombstones, reclaim space from deleted messages within blocks. +5. **MemStore enhancements**: TTL edge cases, limits enforcement, state recovery after restart. + +**Dependencies:** None — foundational track. + +--- + +## Track 2: JetStream Core API (220 tests) + +**Gaps addressed:** Gap 7 (JetStream API — HIGH), Gap 3 (Mirrors/Sources — HIGH) + +**Go test files:** +- `jetstream_test.go`: 173 unmapped +- `jetstream_batching_test.go`: 29 unmapped +- `jetstream_versioning_test.go`: 18 unmapped + +**Current .NET state:** StreamApiHandlers.cs (404 lines), JetStreamApiRouter.cs (203 lines), StreamManager.cs (529 lines), MirrorCoordinator.cs (364 lines), SourceCoordinator.cs (470 lines) + +**Feature work needed:** +1. **Dedup window**: Track `Nats-Msg-Id` headers per stream with configurable TTL. Go: `checkMsgId()`, `storeMsgId()`. +2. **Purge with filter**: `$JS.API.STREAM.PURGE` with subject filter and `keep` parameter. +3. **Snapshot/restore**: API endpoints for stream backup and restore. +4. **Tiered limits**: Per-account JetStream storage limits by tier (R1, R3). +5. **Batch publish**: Handle multi-message publish with atomic success/failure semantics. +6. **Version negotiation**: API version header support for backward compatibility. +7. **Interest retention edge cases**: Correct wildcard/filtered consumer interaction with interest retention. + +**Dependencies:** None — independent of Track 1. + +--- + +## Track 3: JetStream Cluster (296 tests) + +**Gaps addressed:** Gap 2 (JetStream Cluster Coordination — CRITICAL) + +**Go test files:** +- `jetstream_cluster_1_test.go`: 73 unmapped +- `jetstream_cluster_2_test.go`: 92 unmapped +- `jetstream_cluster_3_test.go`: 64 unmapped +- `jetstream_cluster_4_test.go`: 61 unmapped +- `jetstream_cluster_long_test.go`: 6 unmapped + +**Current .NET state:** JetStreamMetaGroup.cs (454 lines), StreamReplicaGroup.cs (300 lines), PlacementEngine.cs (80 lines), JetStreamClusterFixture (existing test infrastructure) + +**Feature work needed:** +1. **Simulated fixture enhancements**: Multi-node behavior simulation (3-node, not just leader), partition simulation (isolate/heal nodes), inflight request dedup. +2. **Multi-server harness** (new ~26 tests): `MultiServerFixture` that starts real NATS.Server.Host processes, configures routes via temp config files, verifies route establishment, tears down on dispose. +3. **Meta-group enhancements**: Peer removal simulation, stream/consumer assignment version tracking, step-down cascading behavior. + +**Testing approach:** +- ~270 tests: Simulated fixtures (extend JetStreamClusterFixture) +- ~26 tests: Real multi-server harness (leader election, partition behavior, route failover) + +**Dependencies:** Track 1 (storage must be solid for cluster storage tests). + +--- + +## Track 4: Consumer Engines (96 tests) + +**Gaps addressed:** Gap 4 (Consumer Delivery Engines — HIGH) + +**Go test files:** +- `jetstream_consumer_test.go`: 96 unmapped + +**Current .NET state:** PushConsumerEngine.cs (264 lines), PullConsumerEngine.cs (300 lines), AckProcessor.cs (225 lines), PriorityGroupManager.cs (102 lines), RedeliveryTracker.cs (92 lines) + +**Feature work needed:** +1. **Backoff schedules**: Configurable per-message redelivery delay arrays in RedeliveryTracker. Go: `getNextRetryDelay()`. +2. **Pause/resume**: Consumer state machine with `Pause(until)`, `Resume()`, advisory publication. +3. **Idle heartbeats**: Timer-based heartbeat generation when no messages delivered within configured interval. +4. **Max deliveries**: Enforcement with configurable dead-letter to `$JS.EVENT.ADVISORY.CONSUMER.MAX_DELIVERIES`. +5. **Replay rate**: Deliver at original publish timestamps for `replay=original` mode. +6. **Filter subject skip tracking**: Skip non-matching messages without counting against pending limits. + +**Dependencies:** None — uses existing storage interface. + +--- + +## Track 5: Auth/JWT/Accounts (105 tests) + +**Gaps addressed:** Gap 9 (Account Management — MEDIUM) + +**Go test files:** +- `jwt_test.go`: 61 unmapped +- `auth_callout_test.go`: 30 unmapped +- `accounts_test.go`: 14 unmapped + +**Current .NET state:** JwtAuthenticator.cs (180 lines), AccountClaims.cs (107 lines), AuthService.cs (172 lines), Account.cs (189 lines), various auth files (~1,266 total) + +**Feature work needed:** +1. **JWT claims depth**: Full operator→account→user trust chain validation. Signing key hierarchy. +2. **Auth callout**: Wire protocol for delegating auth to external service via NATS request/reply. +3. **Account imports/exports**: Runtime enforcement of service/stream exports with authorization checks and cycle detection. +4. **Account resource limits**: Per-account JetStream limits (storage, streams, consumers). + +**Dependencies:** None. + +--- + +## Track 6: Networking (140 tests + 47 super-cluster) + +**Gaps addressed:** Gap 11 (Gateway — MEDIUM), Gap 12 (Leaf Node — MEDIUM), Gap 13 (Routes — MEDIUM) + +**Go test files:** +- `gateway_test.go`: 48 unmapped +- `leafnode_test.go`: 55 unmapped +- `routes_test.go`: 37 unmapped +- `leafnode_proxy_test.go`: 8 unmapped +- `jetstream_leafnode_test.go`: 12 unmapped +- `jetstream_super_cluster_test.go`: 47 unmapped + +**Current .NET state:** GatewayManager.cs (225 lines), GatewayConnection.cs (242 lines), LeafNodeManager.cs (213 lines), LeafConnection.cs (242 lines), RouteManager.cs (269 lines), RouteConnection.cs (289 lines) + +**Feature work needed:** +1. **Gateway interest-only mode**: Switch from flooding to interest-based forwarding after subscription snapshot. +2. **Route pooling**: Multiple connections per peer, round-robin distribution. +3. **Leaf solicited connections**: Outbound leaf connections with retry and backoff. +4. **Route/leaf compression**: Wire S2Codec into route and leaf connection paths. +5. **Super-cluster simulation**: Multi-cluster topology via gateway simulation in test fixtures. +6. **Leaf JetStream domain**: Forward JetStream domain headers through leaf connections. + +**Dependencies:** None. + +--- + +## Track 7: Config/Reload/Monitoring (132 tests) + +**Gaps addressed:** Gap 14 (Configuration — MEDIUM), Gap 10 (Monitoring — MEDIUM) + +**Go test files:** +- `opts_test.go`: 49 unmapped +- `reload_test.go`: 38 unmapped +- `monitor_test.go`: 45 unmapped + +**Current .NET state:** ConfigProcessor.cs (1,023 lines), NatsConfLexer.cs (1,503 lines), ConfigReloader.cs (526 lines), Monitoring/ (1,698 lines) + +**Feature work needed:** +1. **Auth change propagation**: On reload, disconnect clients with revoked credentials. +2. **TLS cert reload**: Reload certificates without restart. +3. **CLI flag coverage**: Parse remaining Go CLI flags. +4. **Monitoring completeness**: Add missing sort options, account filtering to `/connz`. Closed connection ring buffer. +5. **Event payload completeness**: Full JSON payloads for connect/disconnect/auth-error events. + +**Dependencies:** None. + +--- + +## Track 8: Client/Protocol/Misc (112 tests) + +**Gaps addressed:** Gap 5 (Client Protocol — HIGH), misc + +**Go test files:** +- `client_test.go`: 22 unmapped +- `server_test.go`: 28 unmapped +- `sublist_test.go`: 25 unmapped +- `msgtrace_test.go`: 22 unmapped +- `client_proxyproto_test.go`: 23 unmapped +- Smaller: parser (5), closed_conns (6), log (4), errors (2), config_check (2), auth_test (2), jetstream_errors (4), jetstream_jwt (18), jetstream_tpm (5), trust (3), subject_transform (3), nkey (1), ping (1), rate_counter (1), util (6), log (4), mqtt_ex (2), closed_conns (6) + +**Current .NET state:** NatsClient.cs (924 lines), NatsParser.cs (495 lines), SubList.cs (984 lines), MessageTraceContext.cs (686 lines) + +**Feature work needed:** +1. **PROXY protocol**: Parse v1/v2 header for client source IP behind load balancers. +2. **Message trace propagation**: Full trace through publish→match→route→deliver pipeline. +3. **Slow consumer detection**: Pending bytes/messages threshold with eviction advisory. +4. **Adaptive read buffer**: Dynamic resize from 512→65KB based on throughput. +5. **SubList stress tests**: Concurrent insert/match/remove under load. + +**Dependencies:** None. + +--- + +## Execution Strategy + +### Parallelism +- **Independent tracks** (can start immediately): 1, 2, 4, 5, 6, 7, 8 +- **Dependent track**: 3 (waits for Track 1 completion) +- Within each track, subagents work sequentially on feature+test batches + +### DB Update Pattern +After each batch of tests: +```sql +UPDATE go_tests SET status='mapped', dotnet_test='DotNetTestName', dotnet_file='TestFile.cs' + WHERE go_test='GoTestName' AND go_file='go_file.go'; +``` + +### Test Execution +- **During implementation**: `dotnet test --filter "FullyQualifiedName~ClassName"` — targeted only +- **End of all tracks**: `dotnet test` — full suite verification + +### Expected Outcome +- ~224 tests marked `not_applicable` +- ~1,418 tests mapped with passing .NET equivalents +- Final mapping: ~2,666/2,937 (90.8%) +- .NET test count: ~6,300+ (4,886 current + ~1,418 new)