dohertj2/natsdotnet
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Fix E2E test gaps and add comprehensive E2E + parity test suites

Browse Source 
- Fix pull consumer fetch: send original stream subject in HMSG (not inbox)
  so NATS client distinguishes data messages from control messages
- Fix MaxAge expiry: add background timer in StreamManager for periodic pruning
- Fix JetStream wire format: Go-compatible anonymous objects with string enums,
  proper offset-based pagination for stream/consumer list APIs
- Add 42 E2E black-box tests (core messaging, auth, TLS, accounts, JetStream)
- Add ~1000 parity tests across all subsystems (gaps closure)
- Update gap inventory docs to reflect implementation status
This commit is contained in:
Joseph Doherty
2026-03-12 14:09:23 -04:00
parent 79c1ee8776
commit c30e67a69d
226 changed files with 17801 additions and 709 deletions
Download Patch File Download Diff File Expand all files Collapse all files

34
gaps/jetstream.md
View File

@@ -140,12 +140,12 @@ Add rows to the Gap Inventory table below. Group by Go source file. Include the
| Go Symbol | Go File:Line | Status | .NET Equivalent | Notes |
|-----------|:-------------|--------|:----------------|-------|
| JetStreamConfig (struct) | golang/nats-server/server/jetstream.go:42 | PARTIAL | src/NATS.Server/Configuration/JetStreamOptions.cs:5 | .NET has MaxMemoryStore, MaxFileStore, MaxStreams, MaxConsumers, Domain. Missing: SyncInterval, SyncAlways, CompressOK, UniqueTag, Strict fields |
| JetStreamStats (struct) | golang/nats-server/server/jetstream.go:55 | MISSING | — | Server-level usage stats (Memory, Store, ReservedMemory, ReservedStore, Accounts, HAAssets, API) not modeled |
| JetStreamAccountLimits (struct) | golang/nats-server/server/jetstream.go:65 | PARTIAL | src/NATS.Server/Configuration/JetStreamOptions.cs:5 | MaxStreams/MaxConsumers present. Missing: MaxAckPending, MemoryMaxStreamBytes, StoreMaxStreamBytes, MaxBytesRequired, tiered limits |
| JetStreamTier (struct) | golang/nats-server/server/jetstream.go:76 | MISSING | — | Tiered accounting not implemented |
| JetStreamConfig (struct) | golang/nats-server/server/jetstream.go:42 | PORTED | src/NATS.Server/Configuration/JetStreamOptions.cs | Added missing config fields: `SyncInterval`, `SyncAlways`, `CompressOk`, `UniqueTag`, `Strict` (plus parser wiring) |
| JetStreamStats (struct) | golang/nats-server/server/jetstream.go:55 | PORTED | src/NATS.Server/JetStream/JetStreamParityModels.cs (`JetStreamStats`) | Added server-usage model with `Memory`, `Store`, `ReservedMemory`, `ReservedStore`, `Accounts`, `HaAssets`, and `Api` |
| JetStreamAccountLimits (struct) | golang/nats-server/server/jetstream.go:65 | PORTED | src/NATS.Server/JetStream/JetStreamParityModels.cs (`JetStreamAccountLimits`), src/NATS.Server/Configuration/JetStreamOptions.cs | Added missing limits fields: `MaxAckPending`, `MemoryMaxStreamBytes`, `StoreMaxStreamBytes`, `MaxBytesRequired`, and tier map support |
| JetStreamTier (struct) | golang/nats-server/server/jetstream.go:76 | PORTED | src/NATS.Server/JetStream/JetStreamParityModels.cs (`JetStreamTier`) | Added per-tier model (`Name`, `Memory`, `Store`, `Streams`, `Consumers`) |
| JetStreamAccountStats (struct) | golang/nats-server/server/jetstream.go:87 | PARTIAL | src/NATS.Server/JetStream/Api/JetStreamApiResponse.cs:105 | JetStreamAccountInfo has Streams/Consumers counts only. Missing: memory/store usage, tiers, domain, API stats |
| JetStreamAPIStats (struct) | golang/nats-server/server/jetstream.go:95 | MISSING | — | API level/total/errors/inflight stats not tracked |
| JetStreamAPIStats (struct) | golang/nats-server/server/jetstream.go:95 | PORTED | src/NATS.Server/JetStream/JetStreamParityModels.cs (`JetStreamApiStats`) | Added API stats model with `Level`, `Total`, `Errors`, `Inflight` |
| jetStream (internal struct) | golang/nats-server/server/jetstream.go:103 | PARTIAL | src/NATS.Server/JetStream/JetStreamService.cs:11 | JetStreamService covers lifecycle. Missing: apiInflight/apiTotal/apiErrors atomics, memUsed/storeUsed tracking, accounts map, apiSubs, cluster, oos/shuttingDown state |
| jsAccount (internal struct) | golang/nats-server/server/jetstream.go:151 | MISSING | — | Per-account JetStream state (streams map, usage tracking, cluster usage updates) not modeled |
| jsaUsage (internal struct) | golang/nats-server/server/jetstream.go:181 | MISSING | — | Per-account mem/store usage tracking |
@@ -164,12 +164,12 @@ Add rows to the Gap Inventory table below. Group by Go source file. Include the
| enableAllJetStreamServiceImportsAndMappings (Account method) | golang/nats-server/server/jetstream.go:714 | MISSING | — | Per-account service imports and domain mappings |
| configJetStream (Server method) | golang/nats-server/server/jetstream.go:771 | MISSING | — | Per-account JS config (enable/update/disable) |
| configAllJetStreamAccounts (Server method) | golang/nats-server/server/jetstream.go:809 | MISSING | — | Walk all accounts and restore JetStream state |
| JetStreamEnabled (Server method) | golang/nats-server/server/jetstream.go:904 | PARTIAL | src/NATS.Server/JetStream/JetStreamService.cs:48 | IsRunning property equivalent |
| JetStreamEnabled (Server method) | golang/nats-server/server/jetstream.go:904 | PORTED | src/NATS.Server/NatsServer.cs:159 (`JetStreamEnabled`) | Server-level JetStream enabled check now exposed and backed by service running state |
| JetStreamEnabledForDomain (Server method) | golang/nats-server/server/jetstream.go:909 | MISSING | — | Domain-wide JS availability check |
| signalPullConsumers (Server method) | golang/nats-server/server/jetstream.go:930 | MISSING | — | Shutdown signal to R1 pull consumers |
| shutdownJetStream (Server method) | golang/nats-server/server/jetstream.go:977 | PARTIAL | src/NATS.Server/JetStream/JetStreamService.cs:141 | Basic cleanup in DisposeAsync. Missing: account removal, cluster qch signaling |
| JetStreamConfig (Server method) | golang/nats-server/server/jetstream.go:1055 | MISSING | — | Returns copy of current config |
| StoreDir (Server method) | golang/nats-server/server/jetstream.go:1065 | MISSING | — | Returns current StoreDir |
| JetStreamConfig (Server method) | golang/nats-server/server/jetstream.go:1055 | PORTED | src/NATS.Server/NatsServer.cs:161 (`JetStreamConfig`) | Returns a copy of configured JetStream options (store dir, limits, domain) |
| StoreDir (Server method) | golang/nats-server/server/jetstream.go:1065 | PORTED | src/NATS.Server/NatsServer.cs:177 (`StoreDir`) | Server now exposes current configured JetStream store directory |
| JetStreamNumAccounts (Server method) | golang/nats-server/server/jetstream.go:1074 | MISSING | — | Enabled account count |
| JetStreamReservedResources (Server method) | golang/nats-server/server/jetstream.go:1085 | MISSING | — | Reserved mem/store bytes |
| Account.EnableJetStream | golang/nats-server/server/jetstream.go:1107 | MISSING | — | Per-account JS enablement with limits, store dir, cluster usage |
@@ -194,14 +194,14 @@ Add rows to the Gap Inventory table below. Group by Go source file. Include the
| jsAccount.reservedStorage | golang/nats-server/server/jetstream.go:1801 | MISSING | — | Reserved bytes by tier |
| jsAccount.delete | golang/nats-server/server/jetstream.go:2481 | MISSING | — | Delete all JS resources for account |
| dynJetStreamConfig (Server method) | golang/nats-server/server/jetstream.go:2659 | MISSING | — | Dynamic config: 75% sysmem, disk available |
| isValidName | golang/nats-server/server/jetstream.go:2735 | MISSING | — | Name validation (no spaces, wildcards) |
| isValidName | golang/nats-server/server/jetstream.go:2735 | PORTED | src/NATS.Server/JetStream/Validation/JetStreamConfigValidator.cs:10 | `IsValidName` enforces non-empty names, UTF-8 max 255 bytes, and rejects whitespace / `*` / `>`; applied in stream + consumer create paths |
| friendlyBytes | golang/nats-server/server/jetstream.go:2723 | NOT_APPLICABLE | — | Logging helper; .NET has built-in formatting |
| tierName | golang/nats-server/server/jetstream.go:2316 | MISSING | — | Compute tier name from replica count |
| validateJetStreamOptions | golang/nats-server/server/jetstream.go:2767 | MISSING | — | Validates JS options (domain, cluster, etc.) |
| fixCfgMirrorWithDedupWindow | golang/nats-server/server/jetstream.go:2848 | NOT_APPLICABLE | — | Bug fix for legacy config; not needed in new port |
| JetStreamStoreDir (const) | golang/nats-server/server/jetstream.go:2649 | MISSING | — | "jetstream" directory name constant |
| JetStreamMaxStoreDefault (const) | golang/nats-server/server/jetstream.go:2651 | MISSING | — | Default 1TB disk limit |
| JetStreamMaxMemDefault (const) | golang/nats-server/server/jetstream.go:2653 | MISSING | — | Default 256MB mem limit |
| JetStreamStoreDir (const) | golang/nats-server/server/jetstream.go:2649 | PORTED | src/NATS.Server/Configuration/JetStreamOptions.cs:8 | Added Go-parity constant `"jetstream"` |
| JetStreamMaxStoreDefault (const) | golang/nats-server/server/jetstream.go:2651 | PORTED | src/NATS.Server/Configuration/JetStreamOptions.cs:9 | Added Go-parity default max store constant (`1 TiB`) |
| JetStreamMaxMemDefault (const) | golang/nats-server/server/jetstream.go:2653 | PORTED | src/NATS.Server/Configuration/JetStreamOptions.cs:10 | Added Go-parity default max memory constant (`256 MiB`) |
| Stream recovery logic (doStream/doConsumers) | golang/nats-server/server/jetstream.go:1223-1636 | MISSING | — | Full stream/consumer recovery from disk: metafile reading, checksum, encryption, versioning, subject repair |
| keyGen (type) | golang/nats-server/server/jetstream.go:237 | MISSING | — | Key generation function signature for encryption |
| resourcesExceededError (Server method) | golang/nats-server/server/jetstream.go:2743 | MISSING | — | Throttled error logging + meta leader stepdown |
@@ -213,10 +213,10 @@ Add rows to the Gap Inventory table below. Group by Go source file. Include the
|-----------|:-------------|--------|:----------------|-------|
| JSApi* subject constants (50+) | golang/nats-server/server/jetstream_api.go:36-312 | PORTED | src/NATS.Server/JetStream/Api/JetStreamApiSubjects.cs:1 | All major API subjects defined. Minor: some template variants (T-suffixed) not needed in .NET |
| JSAdvisory* prefix constants (25+) | golang/nats-server/server/jetstream_api.go:229-311 | PARTIAL | src/NATS.Server/JetStream/Api/AdvisoryPublisher.cs:1 | Stream create/delete/update, consumer create/delete covered. Missing: snapshot, restore, leader elected, quorum lost, batch abandoned, out-of-storage, server removed, API limit, pause, pinned, unpinned advisory prefixes |
| JSMaxDescriptionLen (const) | golang/nats-server/server/jetstream_api.go:352 | MISSING | — | 4096 byte limit for descriptions |
| JSMaxMetadataLen (const) | golang/nats-server/server/jetstream_api.go:356 | MISSING | — | 128KB metadata map size limit |
| JSMaxNameLen (const) | golang/nats-server/server/jetstream_api.go:360 | MISSING | — | 255 char name length limit |
| JSDefaultRequestQueueLimit (const) | golang/nats-server/server/jetstream_api.go:364 | MISSING | — | 10,000 request queue limit |
| JSMaxDescriptionLen (const) | golang/nats-server/server/jetstream_api.go:352 | PORTED | src/NATS.Server/JetStream/Api/JetStreamApiLimits.cs:10 | Added Go-parity constant and enforced in stream create/update validation (`StreamManager`) |
| JSMaxMetadataLen (const) | golang/nats-server/server/jetstream_api.go:356 | PORTED | src/NATS.Server/JetStream/Api/JetStreamApiLimits.cs:11 | Added Go-parity constant; metadata byte-size helper validates stream/consumer metadata against this limit |
| JSMaxNameLen (const) | golang/nats-server/server/jetstream_api.go:360 | PORTED | src/NATS.Server/JetStream/Api/JetStreamApiLimits.cs:12 | Added Go-parity constant; name validation uses UTF-8 byte length limit through `JetStreamConfigValidator.IsValidName` |
| JSDefaultRequestQueueLimit (const) | golang/nats-server/server/jetstream_api.go:364 | PORTED | src/NATS.Server/JetStream/Api/JetStreamApiLimits.cs:13 | Added Go-parity default request queue limit constant for JetStream API request orchestration |
| ApiResponse (struct) | golang/nats-server/server/jetstream_api.go:369 | PARTIAL | src/NATS.Server/JetStream/Api/JetStreamApiResponse.cs:5 | Type field missing; error structure simplified |
| ApiPaged / ApiPagedRequest (structs) | golang/nats-server/server/jetstream_api.go:395-404 | MISSING | — | Paged API request/response not implemented |
| JSApiAccountInfoResponse | golang/nats-server/server/jetstream_api.go:407 | PARTIAL | src/NATS.Server/JetStream/Api/JetStreamApiResponse.cs:105 | Basic streams/consumers count. Missing: full JetStreamAccountStats embedding |
@@ -1933,3 +1933,5 @@ After porting work is completed:
| 2026-02-25 | JS-5c: jetstream_cluster.go lines 8001-end. 77 symbols. PORTED:8 PARTIAL:13 MISSING:56 | opus |
| 2026-02-25 | File created with LLM analysis instructions | auto |
| 2026-02-25 | JS-1 Core sub-pass: analyzed jetstream.go, jetstream_api.go, jetstream_events.go, jetstream_errors.go, jetstream_versioning.go, jetstream_batching.go. 150+ symbols inventoried. | opus |
| 2026-02-25 | JS core config parity batch: added JetStream default constants (`JetStreamStoreDir`, max store/mem defaults) and server accessors (`JetStreamEnabled`, `JetStreamConfig`, `StoreDir`) with focused tests | codex |
| 2026-02-26 | JS config/model parity batch: extended `JetStreamOptions` (sync/compress/strict/unique-tag and account-limit fields), added parser support for extended jetstream config keys, and added core parity models (`JetStreamStats`, `JetStreamApiStats`, `JetStreamAccountLimits`, `JetStreamTier`) with focused tests | codex |