dohertj2/natsdotnet
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Fix E2E test gaps and add comprehensive E2E + parity test suites

Browse Source 
- Fix pull consumer fetch: send original stream subject in HMSG (not inbox)
  so NATS client distinguishes data messages from control messages
- Fix MaxAge expiry: add background timer in StreamManager for periodic pruning
- Fix JetStream wire format: Go-compatible anonymous objects with string enums,
  proper offset-based pagination for stream/consumer list APIs
- Add 42 E2E black-box tests (core messaging, auth, TLS, accounts, JetStream)
- Add ~1000 parity tests across all subsystems (gaps closure)
- Update gap inventory docs to reflect implementation status
This commit is contained in:
Joseph Doherty
2026-03-12 14:09:23 -04:00
parent 79c1ee8776
commit c30e67a69d
226 changed files with 17801 additions and 709 deletions
Download Patch File Download Diff File Expand all files Collapse all files

46
gaps/auth-and-accounts.md
View File

@@ -100,8 +100,8 @@ Add rows to the Gap Inventory table below. Group by Go source file. Include the
|-----------|:-------------|--------|:----------------|-------|
| Authentication interface | golang/nats-server/server/auth.go:40 | PORTED | src/NATS.Server/Auth/IAuthenticator.cs:7 | Renamed to IAuthenticator; Check(ClientAuthentication) -> Authenticate(ClientAuthContext) |
| ClientAuthentication interface | golang/nats-server/server/auth.go:46 | PORTED | src/NATS.Server/Auth/IAuthenticator.cs:12 | Mapped to ClientAuthContext record |
| NkeyUser struct | golang/nats-server/server/auth.go:62 | PARTIAL | src/NATS.Server/Auth/NKeyUser.cs:3 | Missing: Issued field, AllowedConnectionTypes, ProxyRequired |
| User struct | golang/nats-server/server/auth.go:73 | PARTIAL | src/NATS.Server/Auth/User.cs:3 | Missing: AllowedConnectionTypes, ProxyRequired |
| NkeyUser struct | golang/nats-server/server/auth.go:62 | PORTED | src/NATS.Server/Auth/NKeyUser.cs:3 | Added parity fields `Issued`, `AllowedConnectionTypes`, and `ProxyRequired` |
| User struct | golang/nats-server/server/auth.go:73 | PORTED | src/NATS.Server/Auth/User.cs:3 | Added parity fields `AllowedConnectionTypes` and `ProxyRequired` |
| User.clone() | golang/nats-server/server/auth.go:85 | NOT_APPLICABLE | — | .NET uses immutable init-only records; deep clone not needed |
| NkeyUser.clone() | golang/nats-server/server/auth.go:106 | NOT_APPLICABLE | — | .NET uses immutable init-only records; deep clone not needed |
| SubjectPermission struct | golang/nats-server/server/auth.go:127 | PORTED | src/NATS.Server/Auth/Permissions.cs:10 | |
@@ -111,21 +111,21 @@ Add rows to the Gap Inventory table below. Group by Go source file. Include the
| SubjectPermission.clone() | golang/nats-server/server/auth.go:156 | NOT_APPLICABLE | — | .NET uses immutable records |
| Permissions.clone() | golang/nats-server/server/auth.go:174 | NOT_APPLICABLE | — | .NET uses immutable records |
| checkAuthforWarnings() | golang/nats-server/server/auth.go:196 | MISSING | — | Warns about plaintext passwords at startup |
| assignGlobalAccountToOrphanUsers() | golang/nats-server/server/auth.go:226 | MISSING | — | Auto-assigns global account to users without one |
| validateResponsePermissions() | golang/nats-server/server/auth.go:243 | MISSING | — | Sets defaults for ResponsePermission (MaxMsgs, Expires) |
| assignGlobalAccountToOrphanUsers() | golang/nats-server/server/auth.go:226 | PORTED | src/NATS.Server/Auth/AuthService.cs:176 | `AuthService.Build` now normalizes orphan users/nkeys onto `$G` via `NormalizeUsers`/`NormalizeNKeys`. |
| validateResponsePermissions() | golang/nats-server/server/auth.go:243 | PORTED | src/NATS.Server/Auth/AuthService.cs:222 | Added response-permission normalization: ensures publish allow-list exists, sets `MaxMsgs` default to `DefaultAllowResponseMaxMsgs`, and `Expires` default to `DefaultAllowResponseExpiration` when zero. |
| configureAuthorization() | golang/nats-server/server/auth.go:266 | PARTIAL | src/NATS.Server/Auth/AuthService.cs:30 | AuthService.Build covers the priority chain; missing websocket/mqtt auth config, auth callout account validation |
| buildNkeysAndUsersFromOptions() | golang/nats-server/server/auth.go:325 | PARTIAL | src/NATS.Server/Auth/AuthService.cs:30 | User/NKey map building is in AuthService.Build; missing clone + account resolution + response permission validation |
| buildNkeysAndUsersFromOptions() | golang/nats-server/server/auth.go:325 | PARTIAL | src/NATS.Server/Auth/AuthService.cs:31 | User/NKey map building, clone normalization, orphan account assignment, and response-permission defaulting now occur in `AuthService.Build`; remaining gaps are server-level warnings and broader router/gateway/leaf auth wiring. |
| checkAuthentication() | golang/nats-server/server/auth.go:365 | PARTIAL | src/NATS.Server/Auth/AuthService.cs:97 | Only CLIENT kind is implemented; ROUTER, GATEWAY, LEAF auth missing |
| isClientAuthorized() | golang/nats-server/server/auth.go:382 | PORTED | src/NATS.Server/Auth/AuthService.cs:97 | Core flow matches; missing accountConnectEvent |
| matchesPinnedCert() | golang/nats-server/server/auth.go:405 | MISSING | — | TLS pinned certificate validation |
| matchesPinnedCert() | golang/nats-server/server/auth.go:405 | PORTED | src/NATS.Server/Tls/TlsHelper.cs:132 | `TlsHelper.MatchesPinnedCert(cert, pinned)` implements hash-based pinned-certificate validation; covered by targeted tests in `TlsHelperTests`. |
| processUserPermissionsTemplate() | golang/nats-server/server/auth.go:427 | PORTED | src/NATS.Server/Auth/Jwt/PermissionTemplates.cs:36 | Full template expansion with cartesian product |
| processClientOrLeafAuthentication() | golang/nats-server/server/auth.go:588 | PARTIAL | src/NATS.Server/Auth/AuthService.cs:97 | Core client auth flow ported; missing leaf node auth, proxy check integration, auth callout defer, JWT src/time validation |
| proxyCheck() | golang/nats-server/server/auth.go:1153 | PARTIAL | src/NATS.Server/Auth/ProxyAuthenticator.cs:3 | Basic proxy prefix auth exists; full NKey signature-based proxy verification missing |
| getTLSAuthDCs() | golang/nats-server/server/auth.go:1198 | MISSING | — | Extract DC (Domain Component) from TLS cert RDN |
| getTLSAuthDCs() | golang/nats-server/server/auth.go:1198 | PORTED | src/NATS.Server/Auth/TlsMapAuthenticator.cs:68 | Added DC extraction helper for TLS auth subject matching (`GetTlsAuthDcs`). |
| tlsMapAuthFn type | golang/nats-server/server/auth.go:1218 | NOT_APPLICABLE | — | Go function type; .NET uses delegate/lambda |
| checkClientTLSCertSubject() | golang/nats-server/server/auth.go:1220 | PARTIAL | src/NATS.Server/Auth/TlsMapAuthenticator.cs:9 | Basic DN/CN matching ported; missing email, URI, DNS SAN matching, LDAP DN parsing |
| dnsAltNameLabels() | golang/nats-server/server/auth.go:1316 | MISSING | — | DNS alt name label splitting for TLS |
| dnsAltNameMatches() | golang/nats-server/server/auth.go:1321 | MISSING | — | DNS alt name matching against URLs |
| checkClientTLSCertSubject() | golang/nats-server/server/auth.go:1220 | PARTIAL | src/NATS.Server/Auth/TlsMapAuthenticator.cs:25 | Added DN/CN plus SAN email/DNS/URI matching and DC-augmented RDN matching; LDAP raw-subject DN parsing and full Go callback behavior remain unported. |
| dnsAltNameLabels() | golang/nats-server/server/auth.go:1316 | PORTED | src/NATS.Server/Auth/TlsMapAuthenticator.cs:85 | Added DNS alt-name label splitter with lowercase normalization. |
| dnsAltNameMatches() | golang/nats-server/server/auth.go:1321 | PORTED | src/NATS.Server/Auth/TlsMapAuthenticator.cs:93 | Added RFC6125-style DNS alt-name matching helper (left-most `*` wildcard only). |
| isRouterAuthorized() | golang/nats-server/server/auth.go:1349 | MISSING | — | Cluster route authentication |
| isGatewayAuthorized() | golang/nats-server/server/auth.go:1390 | MISSING | — | Gateway authentication |
| registerLeafWithAccount() | golang/nats-server/server/auth.go:1425 | MISSING | — | Leaf node account registration |
@@ -143,9 +143,9 @@ Add rows to the Gap Inventory table below. Group by Go source file. Include the
| Go Symbol | Go File:Line | Status | .NET Equivalent | Notes |
|-----------|:-------------|--------|:----------------|-------|
| AuthCalloutSubject const | golang/nats-server/server/auth_callout.go:30 | MISSING | — | "$SYS.REQ.USER.AUTH" subject |
| AuthRequestSubject const | golang/nats-server/server/auth_callout.go:31 | MISSING | — | "nats-authorization-request" |
| AuthRequestXKeyHeader const | golang/nats-server/server/auth_callout.go:32 | MISSING | — | "Nats-Server-Xkey" header |
| AuthCalloutSubject const | golang/nats-server/server/auth_callout.go:30 | PORTED | src/NATS.Server/Auth/ExternalAuthCalloutAuthenticator.cs:5 | Added `$SYS.REQ.USER.AUTH` constant |
| AuthRequestSubject const | golang/nats-server/server/auth_callout.go:31 | PORTED | src/NATS.Server/Auth/ExternalAuthCalloutAuthenticator.cs:6 | Added `nats-authorization-request` constant |
| AuthRequestXKeyHeader const | golang/nats-server/server/auth_callout.go:32 | PORTED | src/NATS.Server/Auth/ExternalAuthCalloutAuthenticator.cs:7 | Added `Nats-Server-Xkey` header constant |
| processClientOrLeafCallout() | golang/nats-server/server/auth_callout.go:36 | PARTIAL | src/NATS.Server/Auth/ExternalAuthCalloutAuthenticator.cs:3 | .NET has a simplified external auth callout via IExternalAuthClient interface; Go implementation uses internal NATS messaging ($SYS subjects), JWT encoding/decoding, XKey encryption, replay prevention — all missing from .NET |
| fillClientInfo() | golang/nats-server/server/auth_callout.go:456 | MISSING | — | Fills jwt.ClientInformation for auth callout requests |
| fillConnectOpts() | golang/nats-server/server/auth_callout.go:477 | MISSING | — | Fills jwt.ConnectOptions for auth callout requests |
@@ -181,7 +181,7 @@ Add rows to the Gap Inventory table below. Group by Go source file. Include the
| limits struct | golang/nats-server/server/accounts.go:127 | PARTIAL | src/NATS.Server/Auth/Account.cs:15-19 | MaxConnections, MaxSubscriptions ported; missing MaxPayload, MaxLeafNodeConnections, MaxRemoteConnections |
| sconns struct | golang/nats-server/server/accounts.go:136 | MISSING | — | Remote server connection count tracking |
| streamImport struct | golang/nats-server/server/accounts.go:142 | PORTED | src/NATS.Server/Imports/StreamImport.cs:7 | |
| ClientInfoHdr const | golang/nats-server/server/accounts.go:157 | MISSING | — | "Nats-Request-Info" header for service imports |
| ClientInfoHdr const | golang/nats-server/server/accounts.go:157 | PORTED | src/NATS.Server/Auth/Account.cs:11 | Added `Account.ClientInfoHdr = "Nats-Request-Info"` constant. |
| serviceImport struct | golang/nats-server/server/accounts.go:160 | PORTED | src/NATS.Server/Imports/ServiceImport.cs:6 | |
| serviceRespEntry struct | golang/nats-server/server/accounts.go:187 | MISSING | — | TTL-tracked response entries |
| ServiceRespType enum | golang/nats-server/server/accounts.go:193 | PORTED | src/NATS.Server/Imports/ServiceResponseType.cs:3 | Singleton, Streamed, Chunked |
@@ -227,8 +227,8 @@ Add rows to the Gap Inventory table below. Group by Go source file. Include the
| Account.RemoveMapping() | golang/nats-server/server/accounts.go:810 | MISSING | — | Remove a subject mapping |
| Account.hasMappings() | golang/nats-server/server/accounts.go:839 | MISSING | — | Check for subject mappings |
| Account.selectMappedSubject() | golang/nats-server/server/accounts.go:848 | MISSING | — | Select destination via weighted mapping |
| Account.SubscriptionInterest() | golang/nats-server/server/accounts.go:929 | PARTIAL | src/NATS.Server/Auth/Account.cs:787 | ServiceImportShadowed checks SubList.Match |
| Account.Interest() | golang/nats-server/server/accounts.go:934 | MISSING | — | Returns count of matching subscriptions |
| Account.SubscriptionInterest() | golang/nats-server/server/accounts.go:929 | PORTED | src/NATS.Server/Auth/Account.cs:797 | Added direct parity API: returns `Interest(subject) > 0`. |
| Account.Interest() | golang/nats-server/server/accounts.go:934 | PORTED | src/NATS.Server/Auth/Account.cs:803 | Added matcher-count API backed by `SubList.NumInterest(subject)` (plain + queue). |
| Account.addClient() | golang/nats-server/server/accounts.go:947 | PORTED | src/NATS.Server/Auth/Account.cs:103 | AddClient checks MaxConnections |
| Account.registerLeafNodeCluster() | golang/nats-server/server/accounts.go:986 | MISSING | — | Leaf node cluster registration |
| Account.hasLeafNodeCluster() | golang/nats-server/server/accounts.go:996 | MISSING | — | Check for leaf node cluster |
@@ -268,11 +268,11 @@ Add rows to the Gap Inventory table below. Group by Go source file. Include the
| Account.SetServiceImportSharing() | golang/nats-server/server/accounts.go:1686 | MISSING | — | Enable/disable service import sharing |
| Account.AddServiceImport() | golang/nats-server/server/accounts.go:1715 | PORTED | src/NATS.Server/Auth/Account.cs:338 | |
| Account.NumPendingReverseResponses() | golang/nats-server/server/accounts.go:1721 | PORTED | src/NATS.Server/Auth/Account.cs:772 | ReverseResponseMapCount |
| Account.NumPendingAllResponses() | golang/nats-server/server/accounts.go:1728 | MISSING | — | Total pending across all service imports |
| Account.NumPendingResponses() | golang/nats-server/server/accounts.go:1736 | MISSING | — | Filtered pending response count |
| Account.NumServiceImports() | golang/nats-server/server/accounts.go:1756 | MISSING | — | Count of service imports |
| rsiReason enum | golang/nats-server/server/accounts.go:1763 | MISSING | — | Response service import removal reason |
| Account.removeRespServiceImport() | golang/nats-server/server/accounts.go:1772 | PARTIAL | src/NATS.Server/Imports/ResponseRouter.cs:60 | CleanupResponse exists; missing reason-based tracking/latency |
| Account.NumPendingAllResponses() | golang/nats-server/server/accounts.go:1728 | PORTED | src/NATS.Server/Auth/Account.cs:813 | Added parity API delegating to `NumPendingResponses("")`. |
| Account.NumPendingResponses() | golang/nats-server/server/accounts.go:1736 | PORTED | src/NATS.Server/Auth/Account.cs:820 | Added filtered/aggregate pending-response counter over `Exports.Responses`, keyed by matched service export. |
| Account.NumServiceImports() | golang/nats-server/server/accounts.go:1756 | PORTED | src/NATS.Server/Auth/Account.cs:843 | Added count of configured service import subject keys (`Imports.Services.Count`). |
| rsiReason enum | golang/nats-server/server/accounts.go:1763 | PORTED | src/NATS.Server/Auth/Account.cs:1047 | Added `ResponseServiceImportRemovalReason` enum (`Ok`, `NoDelivery`, `Timeout`). |
| Account.removeRespServiceImport() | golang/nats-server/server/accounts.go:1772 | PARTIAL | src/NATS.Server/Auth/Account.cs:849 | Added `RemoveRespServiceImport(..., reason)` and reason enum wiring; still missing Go's reverse-entry cleanup and reason-driven latency/metrics side effects. |
| Account.getServiceImportForAccountLocked() | golang/nats-server/server/accounts.go:1795 | MISSING | — | Find service import by dest account + subject |
| Account.removeServiceImport() | golang/nats-server/server/accounts.go:1812 | PORTED | src/NATS.Server/Auth/Account.cs:366 | RemoveServiceImport |
| Account.addReverseRespMapEntry() | golang/nats-server/server/accounts.go:1858 | PORTED | src/NATS.Server/Auth/Account.cs:752 | AddReverseRespMapEntry |
@@ -383,5 +383,9 @@ After porting work is completed:
| Date | Change | By |
|------|--------|----|
| 2026-02-26 | Executed auth-and-accounts batch 4: added auth-option normalization in `AuthService.Build` for orphan account assignment (`$G`) and response-permission defaults (`MaxMsgs`/`Expires` + publish allow-list initialization), with targeted tests (`AuthServiceParityBatch4Tests`). Reclassified `assignGlobalAccountToOrphanUsers` and `validateResponsePermissions` to PORTED and updated `buildNkeysAndUsersFromOptions` residual notes. | codex |
| 2026-02-25 | File created with LLM analysis instructions | auto |
| 2026-02-25 | Gap inventory populated: 5 Go files analyzed (7,260 LOC), 239 symbols classified across auth.go, auth_callout.go, nkey.go, jwt.go, accounts.go. Summary: 64 PORTED, 38 PARTIAL, 128 MISSING, 9 NOT_APPLICABLE, 0 DEFERRED | auto |
| 2026-02-25 | Executed auth-and-accounts batch 1: added parity fields to `NKeyUser` and `User`, added auth callout constants to `ExternalAuthCalloutAuthenticator`, added targeted tests (`AuthModelAndCalloutConstantsParityTests`), and reclassified 5 rows (3 MISSING + 2 PARTIAL) to PORTED | codex |
| 2026-02-25 | Executed auth-and-accounts batch 2: added account parity APIs/constants for interest and response/service-import accounting (`ClientInfoHdr`, `SubscriptionInterest`, `Interest`, `NumPendingAllResponses`, `NumPendingResponses`, `NumServiceImports`), introduced `ResponseServiceImportRemovalReason`, added `RemoveRespServiceImport(..., reason)`, and added targeted tests (`AccountResponseAndInterestParityBatch1Tests`). Reclassified 7 rows to PORTED and updated `removeRespServiceImport` notes. | codex |
| 2026-02-25 | Executed auth-and-accounts batch 3: added TLS auth parity helpers (`GetTlsAuthDcs`, `DnsAltNameLabels`, `DnsAltNameMatches`), extended TLS-map auth matching for SAN email/DNS/URI + DC-augmented RDN, validated pinned-cert helper parity (`TlsHelper.MatchesPinnedCert`), and added targeted tests (`TlsMapAuthParityBatch1Tests`). Reclassified 4 rows to PORTED and updated `checkClientTLSCertSubject` notes. | codex |