Commit Graph

28 Commits

Author SHA1 Message Date
Joseph Doherty
4de691c9c5 perf: add FileStore buffered writes, O(1) state tracking, and eliminate redundant per-publish work
Implement Go-parity background flush loop (coalesce 16KB/8ms) in MsgBlock/FileStore,
replace O(n) GetStateAsync with incremental counters, skip PruneExpired/LoadAsync/
PrunePerSubject when not needed, and bypass RAFT for single-replica streams. Fix counter
tracking bugs in RemoveMsg/EraseMsg/TTL expiry and ObjectDisposedException races in
flush loop disposal. FileStore optimizations verified with 3112/3112 JetStream tests
passing; async publish benchmark remains at ~174 msg/s due to E2E protocol path bottleneck.
2026-03-13 03:11:11 -04:00
Joseph Doherty
95e9f0a92e feat: wire remaining E2E gaps — account imports, subject transforms, JWT auth, service latency
Close all 5 server-side wiring gaps so E2E tests pass without skips:
- System events: bridge user-defined system_account to internal $SYS
- Account imports/exports: config parsing + reverse response import for cross-account request-reply
- Subject transforms: parse mappings config block, apply in ProcessMessage
- JWT auth: parse trusted_keys, resolver MEMORY, resolver_preload in config
- Service latency: timestamp on request, publish ServiceLatencyMsg on response
2026-03-12 23:03:12 -04:00
Joseph Doherty
c30e67a69d Fix E2E test gaps and add comprehensive E2E + parity test suites
- Fix pull consumer fetch: send original stream subject in HMSG (not inbox)
  so NATS client distinguishes data messages from control messages
- Fix MaxAge expiry: add background timer in StreamManager for periodic pruning
- Fix JetStream wire format: Go-compatible anonymous objects with string enums,
  proper offset-based pagination for stream/consumer list APIs
- Add 42 E2E black-box tests (core messaging, auth, TLS, accounts, JetStream)
- Add ~1000 parity tests across all subsystems (gaps closure)
- Update gap inventory docs to reflect implementation status
2026-03-12 14:09:23 -04:00
Joseph Doherty
ff07137ea6 feat: add JetStream config change reload (Gap 14.6)
Add ApplyJetStreamConfigChanges to ConfigReloader for detecting MaxMemoryStore,
MaxFileStore, and Domain changes during hot reload. Add Domain property to
JetStreamOptions. Port 10 unit tests covering all change permutations.
2026-02-25 11:48:22 -05:00
Joseph Doherty
5116aed491 feat: add TLS certificate hot-reload for new connections (Gap 14.3)
Add ReloadTlsCertificates(oldOpts, newOpts) returning TlsReloadResult for
path-based cert comparison and validation during config hot-reload. Add 10
targeted tests covering no-change, path detection, missing file, null transitions,
and success cases.
2026-02-25 11:48:11 -05:00
Joseph Doherty
074ff6b287 feat: add cluster config hot reload (Gap 14.4)
Adds ClusterConfigChangeResult and ApplyClusterConfigChanges to ConfigReloader,
comparing route/gateway/leaf URL sets between old and new NatsOptions and reporting
added/removed routes for connection reconciliation on hot reload.
2026-02-25 11:47:02 -05:00
Joseph Doherty
42e072ad71 feat: add auth change propagation to existing connections (Gap 14.2)
Add PropagateAuthChanges to ConfigReloader that compares Users, Accounts,
and Authorization token between old and new NatsOptions, returning an
AuthChangeResult describing which auth fields changed for connection re-evaluation.
2026-02-25 11:46:28 -05:00
Joseph Doherty
e09835ca70 feat(config): add SIGHUP signal handler and config reload validation
Implements SignalHandler (PosixSignalRegistration for SIGHUP) and
ReloadFromOptionsAsync/ReloadFromOptionsResult on ConfigReloader for
in-memory options comparison without reading a config file.
Ports Go server/signal_unix.go handleSignals and server/reload.go Reload.
2026-02-25 02:54:13 -05:00
Joseph Doherty
1a3fe91611 feat: enhance ConfigProcessor & add 110 Go-parity opts tests (Task 22)
Port configuration parsing for NKey users, gateway remotes, leaf node
remotes, auth timeout, write_deadline, websocket ping_interval, and
token+users conflict validation. Add RemoteGatewayOptions, enhanced
LeafNodeOptions with remotes support.

110 new tests ported from opts_test.go.
2026-02-24 20:17:48 -05:00
Joseph Doherty
37d3cc29ea feat(networking): add leaf subject filtering and port networking Go tests (D6+D7)
D6: Add ExportSubjects/ImportSubjects allow-lists to LeafHubSpokeMapper alongside
existing DenyExports/DenyImports deny-lists. When an allow-list is non-empty, subjects
must match at least one allow pattern; deny always takes precedence. Updated
LeafNodeOptions, LeafHubSpokeMapper (5-arg constructor), and LeafNodeManager to wire
through the new allow-lists. Added 13 new unit + integration tests covering allow-list
semantics, deny precedence, bidirectional filtering, and wire-level propagation.

D7: Existing NetworkingGoParityTests.cs (50 tests) covers gateway interest mode,
route pool accounting, and leaf node connections. Parity DB already up to date.
2026-02-24 16:07:33 -05:00
Joseph Doherty
02531dda58 feat(config+ws): add TLS cert reload, WS compression negotiation, WS JWT auth (E9+E10+E11)
E9: TLS Certificate Reload
- Add TlsCertificateProvider with Interlocked-swappable cert field
- New connections get current cert, existing connections keep theirs
- ConfigReloader.ReloadTlsCertificate rebuilds SslServerAuthenticationOptions
- NatsServer.ApplyConfigChanges triggers TLS reload on TLS config changes
- 11 tests covering cert swap, versioning, thread safety, config diff

E10: WebSocket Compression Negotiation (RFC 7692)
- Add WsDeflateNegotiator to parse Sec-WebSocket-Extensions parameters
- Parse server_no_context_takeover, client_no_context_takeover,
  server_max_window_bits, client_max_window_bits
- WsDeflateParams record struct with ToResponseHeaderValue()
- NATS always enforces no_context_takeover (matching Go server)
- WsUpgrade returns negotiated WsDeflateParams in upgrade result
- 22 tests covering parameter parsing, clamping, response headers

E11: WebSocket JWT Authentication
- Extract JWT from Authorization header (Bearer token), cookie, or ?jwt= query param
- Priority: Authorization header > cookie > query parameter
- WsUpgrade.TryUpgradeAsync now parses query string from request URI
- Add FailUnauthorizedAsync for 401 responses
- 24 tests covering all JWT extraction sources and priority ordering
2026-02-24 16:03:46 -05:00
Joseph Doherty
c6ecbbfbcc feat(config): add system account, SIGHUP reload, and auth change propagation (E6+E7+E8)
E6: Add IsSystemAccount property to Account, mark $SYS account as system,
add IsSystemSubject/IsSubscriptionAllowed/GetSubListForSubject helpers to
route $SYS.> subjects to the system account's SubList and block non-system
accounts from subscribing.

E7: Add ConfigReloader.ReloadAsync and ApplyDiff for structured async reload,
add ConfigReloadResult/ConfigApplyResult types. SIGHUP handler already wired
via PosixSignalRegistration in HandleSignals.

E8: Add PropagateAuthChanges to re-evaluate connected clients after auth
config reload, disconnecting clients whose credentials no longer pass
authentication with -ERR 'Authorization Violation'.
2026-02-24 15:48:48 -05:00
Joseph Doherty
efd053ba60 feat(networking): expand gateway reply mapper and add leaf solicited connections (D4+D5)
D4: Add hash segment support to ReplyMapper (_GR_.{cluster}.{hash}.{reply}),
FNV-1a ComputeReplyHash, TryExtractClusterId/Hash, legacy format compat.
D5: Add ConnectSolicitedAsync with exponential backoff (1s-60s cap),
JetStreamDomain propagation in LEAF handshake, LeafNodeOptions.JetStreamDomain.
2026-02-24 15:22:24 -05:00
Joseph Doherty
2c9683e7aa feat: upgrade JetStreamService to lifecycle orchestrator
Implements enableJetStream() semantics from golang/nats-server/server/jetstream.go:414-523.

- JetStreamService.StartAsync(): validates config, creates store directory
  (including nested paths via Directory.CreateDirectory), registers all
  $JS.API.> subjects, logs startup stats; idempotent on double-start
- JetStreamService.DisposeAsync(): clears registered subjects, marks not running
- New properties: RegisteredApiSubjects, MaxStreams, MaxConsumers, MaxMemory, MaxStore
- JetStreamOptions: adds MaxStreams and MaxConsumers limits (0 = unlimited)
- FileStoreConfig: removes duplicate StoreCipher/StoreCompression enum declarations
  now that AeadEncryptor.cs owns them; updates defaults to NoCipher/NoCompression
- FileStoreOptions/FileStore: align enum member names with AeadEncryptor.cs
  (NoCipher, NoCompression, S2Compression) to fix cross-task naming conflict
- 13 new tests in JetStreamServiceOrchestrationTests covering all lifecycle paths
2026-02-24 06:03:46 -05:00
Joseph Doherty
1c0fc8fc11 feat: add runtime profiling parity and close config runtime drift 2026-02-23 14:56:27 -05:00
Joseph Doherty
2b64d762f6 feat: execute full-repo remaining parity closure plan 2026-02-23 13:08:52 -05:00
Joseph Doherty
8bce096f55 feat: complete final jetstream parity transport and runtime baselines 2026-02-23 11:04:43 -05:00
Joseph Doherty
a8985ecb1a Merge branch 'codex/jetstream-full-parity-executeplan' into main
# Conflicts:
#	differences.md
#	docs/plans/2026-02-23-jetstream-full-parity-plan.md
#	src/NATS.Server/Auth/Account.cs
#	src/NATS.Server/Configuration/ConfigProcessor.cs
#	src/NATS.Server/Monitoring/VarzHandler.cs
#	src/NATS.Server/NatsClient.cs
#	src/NATS.Server/NatsOptions.cs
#	src/NATS.Server/NatsServer.cs
2026-02-23 08:53:44 -05:00
Joseph Doherty
6c83f12e5c feat: add reload semantics for cluster and jetstream options 2026-02-23 06:23:34 -05:00
Joseph Doherty
a661e641c6 feat: add mqtt config model and parser for all Go MQTTOpts fields 2026-02-23 05:57:28 -05:00
Joseph Doherty
5f98e53d62 feat: add route handshake lifecycle 2026-02-23 05:46:59 -05:00
Joseph Doherty
44d426a7c5 feat: parse cluster and jetstream config blocks 2026-02-23 05:43:04 -05:00
Joseph Doherty
d21243bc8a feat: add config reloader with diff, validate, and CLI merge
Port of Go server/reload.go option interface and diffing logic. Compares
NatsOptions property-by-property to detect changes, tags each with category
flags (logging, auth, TLS, non-reloadable), validates that non-reloadable
options (Host, Port, ServerName) are not changed at runtime, and provides
MergeCliOverrides to ensure CLI flags always take precedence over config
file values during hot reload.
2026-02-23 04:53:25 -05:00
Joseph Doherty
8a2ded8e48 feat: add config processor mapping parsed config to NatsOptions
Port of Go server/opts.go processConfigFileLine switch. Maps parsed
NATS config dictionaries to NatsOptions fields including:
- Core options (port, host, server_name, limits, ping, write_deadline)
- Logging (debug, trace, logfile, log rotation)
- Authorization (single user, users array with permissions)
- TLS (cert/key/ca, verify, pinned_certs, handshake_first)
- Monitoring (http_port, https_port, http/https listen, base_path)
- Lifecycle (lame_duck_duration/grace_period)
- Server tags, file paths, system account options

Includes error collection (not fail-fast), duration parsing (ms/s/m/h
strings and numeric seconds), host:port listen parsing, and 56 tests
covering all config sections plus validation edge cases.
2026-02-23 04:47:54 -05:00
Joseph Doherty
5219f77f9b fix: add include depth limit, fix PopContext guard, add SetValue fallback
- Add MaxIncludeDepth = 10 constant and thread _includeDepth through ParserState
  constructors, ProcessInclude, ParseFile (private overload), and ParseEnvValue
  to prevent StackOverflowException from recursive includes
- Fix PopContext to check _ctxs.Count <= 1 instead of == 0 so the root context
  is never popped, replacing silent crash with clear InvalidOperationException
- Add else throw in SetValue so unknown context types surface as bugs rather
  than silently dropping values
2026-02-23 04:42:37 -05:00
Joseph Doherty
9f66ef72c6 feat: add NATS config file parser (port of Go conf/parse.go)
Implements NatsConfParser with Parse, ParseFile, and ParseFileWithDigest
methods. Supports nested maps/arrays, variable resolution with block
scoping and environment fallback, bcrypt password literals, integer
suffix multipliers, include directives, and cycle detection.
2026-02-23 04:35:46 -05:00
Joseph Doherty
ae043136a1 fix: address lexer code review findings (newline handling, emit cleanup, null guard) 2026-02-23 04:30:36 -05:00
Joseph Doherty
9fff5709c4 feat: add NATS config file lexer (port of Go conf/lex.go)
Port the NATS configuration file lexer from Go's conf/lex.go to C#.
The lexer is a state-machine tokenizer that supports the NATS config
format: key-value pairs with =, :, or whitespace separators; nested
maps {}; arrays []; single and double quoted strings with escape
sequences; block strings (); variables $VAR; include directives;
comments (# and //); booleans; integers with size suffixes (kb, mb, gb);
floats; ISO8601 datetimes; and IP addresses.
2026-02-23 04:20:56 -05:00