# Config File Parsing & Hot Reload Implementation Plan > **For Claude:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:executing-plans to implement this plan task-by-task. **Goal:** Port the Go NATS config file parser and hot reload system to .NET, resolving the two remaining high-priority gaps in differences.md. **Architecture:** A state-machine lexer tokenizes the NATS config format, a parser builds `Dictionary` from the token stream, a config processor maps keys to `NatsOptions` fields, and a reloader diffs old/new options on SIGHUP to apply changes without restart. **Tech Stack:** .NET 10 / C# 14, xUnit 3, Shouldly, System.IO.Pipelines (existing), Serilog (existing). --- ### Task 1: Token Types and Lexer Infrastructure **Files:** - Create: `src/NATS.Server/Configuration/NatsConfToken.cs` - Create: `src/NATS.Server/Configuration/NatsConfLexer.cs` - Test: `tests/NATS.Server.Tests/NatsConfLexerTests.cs` **Step 1: Write failing tests for basic token types** ```csharp // tests/NATS.Server.Tests/NatsConfLexerTests.cs using NATS.Server.Configuration; namespace NATS.Server.Tests; public class NatsConfLexerTests { [Fact] public void Lex_SimpleKeyStringValue_ReturnsKeyAndString() { var tokens = NatsConfLexer.Tokenize("foo = \"bar\"").ToList(); tokens[0].Type.ShouldBe(TokenType.Key); tokens[0].Value.ShouldBe("foo"); tokens[1].Type.ShouldBe(TokenType.String); tokens[1].Value.ShouldBe("bar"); tokens[2].Type.ShouldBe(TokenType.Eof); } [Fact] public void Lex_SingleQuotedString_ReturnsString() { var tokens = NatsConfLexer.Tokenize("foo = 'bar'").ToList(); tokens[1].Type.ShouldBe(TokenType.String); tokens[1].Value.ShouldBe("bar"); } [Fact] public void Lex_IntegerValue_ReturnsInteger() { var tokens = NatsConfLexer.Tokenize("port = 4222").ToList(); tokens[0].Type.ShouldBe(TokenType.Key); tokens[0].Value.ShouldBe("port"); tokens[1].Type.ShouldBe(TokenType.Integer); tokens[1].Value.ShouldBe("4222"); } [Fact] public void Lex_IntegerWithSuffix_ReturnsInteger() { var tokens = NatsConfLexer.Tokenize("size = 64mb").ToList(); tokens[1].Type.ShouldBe(TokenType.Integer); tokens[1].Value.ShouldBe("64mb"); } [Fact] public void Lex_BooleanValues_ReturnsBool() { foreach (var val in new[] { "true", "false", "yes", "no", "on", "off" }) { var tokens = NatsConfLexer.Tokenize($"flag = {val}").ToList(); tokens[1].Type.ShouldBe(TokenType.Bool); } } [Fact] public void Lex_FloatValue_ReturnsFloat() { var tokens = NatsConfLexer.Tokenize("rate = 2.5").ToList(); tokens[1].Type.ShouldBe(TokenType.Float); tokens[1].Value.ShouldBe("2.5"); } [Fact] public void Lex_NegativeNumber_ReturnsInteger() { var tokens = NatsConfLexer.Tokenize("offset = -10").ToList(); tokens[1].Type.ShouldBe(TokenType.Integer); tokens[1].Value.ShouldBe("-10"); } [Fact] public void Lex_DatetimeValue_ReturnsDatetime() { var tokens = NatsConfLexer.Tokenize("ts = 2024-01-15T10:30:00Z").ToList(); tokens[1].Type.ShouldBe(TokenType.DateTime); } [Fact] public void Lex_HashComment_IsIgnored() { var tokens = NatsConfLexer.Tokenize("# this is a comment\nfoo = 1").ToList(); // First meaningful token should be the key var keys = tokens.Where(t => t.Type == TokenType.Key).ToList(); keys.Count.ShouldBe(1); keys[0].Value.ShouldBe("foo"); } [Fact] public void Lex_SlashComment_IsIgnored() { var tokens = NatsConfLexer.Tokenize("// comment\nfoo = 1").ToList(); var keys = tokens.Where(t => t.Type == TokenType.Key).ToList(); keys.Count.ShouldBe(1); } [Fact] public void Lex_MapBlock_ReturnsMapStartEnd() { var tokens = NatsConfLexer.Tokenize("auth { user: admin }").ToList(); tokens[0].Type.ShouldBe(TokenType.Key); tokens[0].Value.ShouldBe("auth"); tokens[1].Type.ShouldBe(TokenType.MapStart); tokens[2].Type.ShouldBe(TokenType.Key); tokens[2].Value.ShouldBe("user"); tokens[3].Type.ShouldBe(TokenType.String); tokens[3].Value.ShouldBe("admin"); tokens[4].Type.ShouldBe(TokenType.MapEnd); } [Fact] public void Lex_Array_ReturnsArrayStartEnd() { var tokens = NatsConfLexer.Tokenize("items = [1, 2, 3]").ToList(); tokens[1].Type.ShouldBe(TokenType.ArrayStart); tokens[2].Type.ShouldBe(TokenType.Integer); tokens[2].Value.ShouldBe("1"); tokens[5].Type.ShouldBe(TokenType.ArrayEnd); } [Fact] public void Lex_Variable_ReturnsVariable() { var tokens = NatsConfLexer.Tokenize("secret = $MY_VAR").ToList(); tokens[1].Type.ShouldBe(TokenType.Variable); tokens[1].Value.ShouldBe("MY_VAR"); } [Fact] public void Lex_Include_ReturnsInclude() { var tokens = NatsConfLexer.Tokenize("include \"auth.conf\"").ToList(); tokens[0].Type.ShouldBe(TokenType.Include); tokens[0].Value.ShouldBe("auth.conf"); } [Fact] public void Lex_EscapeSequences_AreProcessed() { var tokens = NatsConfLexer.Tokenize("msg = \"hello\\tworld\\n\"").ToList(); tokens[1].Type.ShouldBe(TokenType.String); tokens[1].Value.ShouldBe("hello\tworld\n"); } [Fact] public void Lex_HexEscape_IsProcessed() { var tokens = NatsConfLexer.Tokenize("val = \"\\x41\\x42\"").ToList(); tokens[1].Value.ShouldBe("AB"); } [Fact] public void Lex_ColonSeparator_Works() { var tokens = NatsConfLexer.Tokenize("foo: bar").ToList(); tokens[0].Type.ShouldBe(TokenType.Key); tokens[1].Type.ShouldBe(TokenType.String); } [Fact] public void Lex_WhitespaceSeparator_Works() { var tokens = NatsConfLexer.Tokenize("foo bar").ToList(); tokens[0].Type.ShouldBe(TokenType.Key); tokens[1].Type.ShouldBe(TokenType.String); } [Fact] public void Lex_SemicolonTerminator_IsHandled() { var tokens = NatsConfLexer.Tokenize("foo = 1; bar = 2").ToList(); var keys = tokens.Where(t => t.Type == TokenType.Key).ToList(); keys.Count.ShouldBe(2); } [Fact] public void Lex_EmptyInput_ReturnsEof() { var tokens = NatsConfLexer.Tokenize("").ToList(); tokens.Count.ShouldBe(1); tokens[0].Type.ShouldBe(TokenType.Eof); } [Fact] public void Lex_BlockString_ReturnsString() { // Block strings delimited by ( ... ) where ) is on its own line var input = "desc (\nthis is\na block\n)\n"; var tokens = NatsConfLexer.Tokenize(input).ToList(); tokens[0].Type.ShouldBe(TokenType.Key); tokens[1].Type.ShouldBe(TokenType.String); } [Fact] public void Lex_IPAddress_ReturnsString() { var tokens = NatsConfLexer.Tokenize("host = 127.0.0.1").ToList(); tokens[1].Type.ShouldBe(TokenType.String); tokens[1].Value.ShouldBe("127.0.0.1"); } [Fact] public void Lex_TrackLineNumbers() { var tokens = NatsConfLexer.Tokenize("a = 1\nb = 2\nc = 3").ToList(); tokens[0].Line.ShouldBe(1); // a tokens[2].Line.ShouldBe(2); // b tokens[4].Line.ShouldBe(3); // c } [Fact] public void Lex_UnterminatedString_ReturnsError() { var tokens = NatsConfLexer.Tokenize("foo = \"unterminated").ToList(); tokens.ShouldContain(t => t.Type == TokenType.Error); } [Fact] public void Lex_StringStartingWithDigit_TreatedAsString() { // Go: `foo = 3xyz` → string "3xyz" var tokens = NatsConfLexer.Tokenize("foo = 3xyz").ToList(); tokens[1].Type.ShouldBe(TokenType.String); tokens[1].Value.ShouldBe("3xyz"); } } ``` **Step 2: Run tests to verify they fail** Run: `dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~NatsConfLexerTests" -v normal` Expected: FAIL — types don't exist yet **Step 3: Implement token types** ```csharp // src/NATS.Server/Configuration/NatsConfToken.cs namespace NATS.Server.Configuration; public enum TokenType { Error, Eof, Key, String, Bool, Integer, Float, DateTime, ArrayStart, ArrayEnd, MapStart, MapEnd, Variable, Include, Comment, } public readonly record struct Token(TokenType Type, string Value, int Line, int Position); ``` **Step 4: Implement the lexer** Port Go `conf/lex.go` to C#. Create `src/NATS.Server/Configuration/NatsConfLexer.cs`. Key mapping from Go → C#: - `stateFn func(lx *lexer) stateFn` → `delegate LexState? LexState(Lexer lx)` - `chan item` → `List` (no need for channel, we collect eagerly) - `lx.next()` / `lx.backup()` / `lx.peek()` → same pattern on `ReadOnlySpan` or string with position tracking - `lx.emit()` → add to `List` - `lx.push()` / `lx.pop()` → `Stack` - `lx.stringParts` → `List` for escape sequence assembly - `isWhitespace`, `isNL`, `isKeySeparator`, `isNumberSuffix` → static helper methods The lexer is ~400 lines. Follow the Go state machine exactly: - `lexTop` → `lexKeyStart` → `lexKey` → `lexKeyEnd` → `lexValue` - `lexValue` dispatches to: `lexArrayValue`, `lexMapKeyStart`, `lexQuotedString`, `lexDubQuotedString`, `lexNegNumberStart`, `lexBlock`, `lexNumberOrDateOrStringOrIPStart`, `lexString` - `lexStringEscape` handles `\t`, `\n`, `\r`, `\"`, `\\`, `\xHH` - `lexBlock` looks for `)` on a line by itself - `lexConvenientNumber` handles size suffixes (`k`, `kb`, `mb`, `gb`, etc.) Public API: `static IReadOnlyList Tokenize(string input)` **Step 5: Run tests to verify they pass** Run: `dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~NatsConfLexerTests" -v normal` Expected: PASS **Step 6: Commit** ```bash git add src/NATS.Server/Configuration/NatsConfToken.cs src/NATS.Server/Configuration/NatsConfLexer.cs tests/NATS.Server.Tests/NatsConfLexerTests.cs git commit -m "feat: add NATS config file lexer (port of Go conf/lex.go)" ``` --- ### Task 2: Config Parser **Files:** - Create: `src/NATS.Server/Configuration/NatsConfParser.cs` - Test: `tests/NATS.Server.Tests/NatsConfParserTests.cs` **Step 1: Write failing tests** ```csharp // tests/NATS.Server.Tests/NatsConfParserTests.cs using NATS.Server.Configuration; namespace NATS.Server.Tests; public class NatsConfParserTests { [Fact] public void Parse_SimpleTopLevel_ReturnsCorrectTypes() { var result = NatsConfParser.Parse("foo = '1'; bar = 2.2; baz = true; boo = 22"); result["foo"].ShouldBe("1"); result["bar"].ShouldBe(2.2); result["baz"].ShouldBe(true); result["boo"].ShouldBe(22L); } [Fact] public void Parse_Booleans_AllVariants() { foreach (var (input, expected) in new[] { ("true", true), ("TRUE", true), ("yes", true), ("on", true), ("false", false), ("FALSE", false), ("no", false), ("off", false) }) { var result = NatsConfParser.Parse($"flag = {input}"); result["flag"].ShouldBe(expected); } } [Fact] public void Parse_IntegerWithSuffix_AppliesMultiplier() { var result = NatsConfParser.Parse("a = 1k; b = 2mb; c = 3gb; d = 4kb"); result["a"].ShouldBe(1000L); result["b"].ShouldBe(2L * 1024 * 1024); result["c"].ShouldBe(3L * 1000 * 1000 * 1000); result["d"].ShouldBe(4L * 1024); } [Fact] public void Parse_NestedMap_ReturnsDictionary() { var result = NatsConfParser.Parse("auth { user: admin, pass: secret }"); var auth = result["auth"].ShouldBeOfType>(); auth["user"].ShouldBe("admin"); auth["pass"].ShouldBe("secret"); } [Fact] public void Parse_Array_ReturnsList() { var result = NatsConfParser.Parse("items = [1, 2, 3]"); var items = result["items"].ShouldBeOfType>(); items.Count.ShouldBe(3); items[0].ShouldBe(1L); } [Fact] public void Parse_Variable_ResolvesFromContext() { var result = NatsConfParser.Parse("index = 22\nfoo = $index"); result["foo"].ShouldBe(22L); } [Fact] public void Parse_NestedVariable_UsesBlockScope() { var input = "index = 22\nnest {\n index = 11\n foo = $index\n}\nbar = $index"; var result = NatsConfParser.Parse(input); var nest = result["nest"].ShouldBeOfType>(); nest["foo"].ShouldBe(11L); // inner scope result["bar"].ShouldBe(22L); // outer scope } [Fact] public void Parse_EnvironmentVariable_ResolvesFromEnv() { Environment.SetEnvironmentVariable("NATS_TEST_VAR_12345", "hello"); try { var result = NatsConfParser.Parse("val = $NATS_TEST_VAR_12345"); result["val"].ShouldBe("hello"); } finally { Environment.SetEnvironmentVariable("NATS_TEST_VAR_12345", null); } } [Fact] public void Parse_UndefinedVariable_Throws() { Should.Throw(() => NatsConfParser.Parse("val = $UNDEFINED_VAR_XYZZY_99999")); } [Fact] public void Parse_IncludeDirective_MergesFile() { var dir = Path.Combine(Path.GetTempPath(), $"nats_test_{Guid.NewGuid():N}"); Directory.CreateDirectory(dir); try { File.WriteAllText(Path.Combine(dir, "main.conf"), "port = 4222\ninclude \"sub.conf\""); File.WriteAllText(Path.Combine(dir, "sub.conf"), "host = \"localhost\""); var result = NatsConfParser.ParseFile(Path.Combine(dir, "main.conf")); result["port"].ShouldBe(4222L); result["host"].ShouldBe("localhost"); } finally { Directory.Delete(dir, true); } } [Fact] public void Parse_MultipleKeySeparators_AllWork() { var r1 = NatsConfParser.Parse("a = 1"); var r2 = NatsConfParser.Parse("a : 1"); var r3 = NatsConfParser.Parse("a 1"); r1["a"].ShouldBe(1L); r2["a"].ShouldBe(1L); r3["a"].ShouldBe(1L); } [Fact] public void Parse_ErrorOnInvalidInput_Throws() { Should.Throw(() => NatsConfParser.Parse("= invalid")); } [Fact] public void Parse_CommentsInsideBlocks_AreIgnored() { var input = "auth {\n # comment\n user: admin\n // another comment\n pass: secret\n}"; var result = NatsConfParser.Parse(input); var auth = result["auth"].ShouldBeOfType>(); auth["user"].ShouldBe("admin"); auth["pass"].ShouldBe("secret"); } [Fact] public void Parse_ArrayOfMaps_Works() { var input = "users = [\n { user: alice, pass: pw1 }\n { user: bob, pass: pw2 }\n]"; var result = NatsConfParser.Parse(input); var users = result["users"].ShouldBeOfType>(); users.Count.ShouldBe(2); var first = users[0].ShouldBeOfType>(); first["user"].ShouldBe("alice"); } [Fact] public void Parse_BcryptPassword_HandledAsString() { // Bcrypt strings starting with $2a$ should be preserved var input = "pass = $2a$04$P/.bd.7unw9Ew7yWJqXsl.f4oNRLQGvadEL2YnqQXbbb.IVQajRdK"; var result = NatsConfParser.Parse(input); ((string)result["pass"]!).ShouldStartWith("$2a$"); } [Fact] public void ParseFile_WithDigest_ReturnsStableHash() { var dir = Path.Combine(Path.GetTempPath(), $"nats_test_{Guid.NewGuid():N}"); Directory.CreateDirectory(dir); try { var conf = Path.Combine(dir, "test.conf"); File.WriteAllText(conf, "port = 4222\nhost = \"localhost\""); var (result, digest) = NatsConfParser.ParseFileWithDigest(conf); result["port"].ShouldBe(4222L); digest.ShouldStartWith("sha256:"); digest.Length.ShouldBeGreaterThan(10); // Same input → same digest var (_, digest2) = NatsConfParser.ParseFileWithDigest(conf); digest2.ShouldBe(digest); } finally { Directory.Delete(dir, true); } } } ``` **Step 2: Run tests to verify they fail** Run: `dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~NatsConfParserTests" -v normal` Expected: FAIL — `NatsConfParser` doesn't exist **Step 3: Implement the parser** Port Go `conf/parse.go` to C#. Create `src/NATS.Server/Configuration/NatsConfParser.cs`. Key mapping from Go → C#: - `parser.mapping` → `Dictionary` - `parser.ctx` / `parser.ctxs` → `Stack` for context (map or list) - `parser.keys` → `Stack` for key tracking - `parser.fp` → `string` file path for relative `include` resolution - `processItem` switch on `TokenType` → same pattern - Integer suffix processing: `"k"→×1000`, `"kb"/"ki"/"kib"→×1024`, etc. (match Go exactly) - Variable lookup: walk context stack bottom-up, then `Environment.GetEnvironmentVariable()` - Bcrypt special case: `$2a$...` → treat as literal string - Include: `ParseFile(Path.Combine(baseDir, includePath))`, merge into current context - Digest: `SHA256.HashData()` on JSON serialization of result Public API: - `static Dictionary Parse(string data)` - `static Dictionary ParseFile(string filePath)` - `static (Dictionary Config, string Digest) ParseFileWithDigest(string filePath)` **Step 4: Run tests to verify they pass** Run: `dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~NatsConfParserTests" -v normal` Expected: PASS **Step 5: Commit** ```bash git add src/NATS.Server/Configuration/NatsConfParser.cs tests/NATS.Server.Tests/NatsConfParserTests.cs git commit -m "feat: add NATS config file parser (port of Go conf/parse.go)" ``` --- ### Task 3: New NatsOptions Fields **Files:** - Modify: `src/NATS.Server/NatsOptions.cs` - Modify: `src/NATS.Server/NatsServer.cs:23` (MaxClosedClients constant → use options) - Modify: `tests/NATS.Server.Tests/NatsOptionsTests.cs` **Step 1: Write failing tests for new defaults** ```csharp // Add to tests/NATS.Server.Tests/NatsOptionsTests.cs [Fact] public void New_fields_have_correct_defaults() { var opts = new NatsOptions(); opts.ClientAdvertise.ShouldBeNull(); opts.TraceVerbose.ShouldBeFalse(); opts.MaxTracedMsgLen.ShouldBe(0); opts.DisableSublistCache.ShouldBeFalse(); opts.ConnectErrorReports.ShouldBe(3600); opts.ReconnectErrorReports.ShouldBe(1); opts.NoHeaderSupport.ShouldBeFalse(); opts.MaxClosedClients.ShouldBe(10_000); opts.NoSystemAccount.ShouldBeFalse(); opts.SystemAccount.ShouldBeNull(); } ``` **Step 2: Run tests to verify they fail** Run: `dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~NatsOptionsTests.New_fields" -v normal` Expected: FAIL — properties don't exist **Step 3: Add new fields to NatsOptions** Add to `src/NATS.Server/NatsOptions.cs` (after the existing `ProfPort` property, before TLS section): ```csharp // Extended options for Go parity public string? ClientAdvertise { get; set; } public bool TraceVerbose { get; set; } public int MaxTracedMsgLen { get; set; } public bool DisableSublistCache { get; set; } public int ConnectErrorReports { get; set; } = 3600; public int ReconnectErrorReports { get; set; } = 1; public bool NoHeaderSupport { get; set; } public int MaxClosedClients { get; set; } = 10_000; public bool NoSystemAccount { get; set; } public string? SystemAccount { get; set; } ``` Also add a `HashSet` for CLI flag tracking: ```csharp // Tracks which fields were set via CLI flags (for reload precedence) public HashSet InCmdLine { get; } = []; ``` **Step 4: Update NatsServer.cs to use options.MaxClosedClients** In `src/NATS.Server/NatsServer.cs`: - Remove line 23: `private const int MaxClosedClients = 10_000;` - Change line 650 reference: `while (_closedClients.Count > MaxClosedClients)` → `while (_closedClients.Count > _options.MaxClosedClients)` **Step 5: Run tests to verify they pass** Run: `dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~NatsOptionsTests" -v normal` Expected: PASS Run: `dotnet test tests/NATS.Server.Tests -v normal` Expected: PASS (all existing tests still pass) **Step 6: Commit** ```bash git add src/NATS.Server/NatsOptions.cs src/NATS.Server/NatsServer.cs tests/NATS.Server.Tests/NatsOptionsTests.cs git commit -m "feat: add new NatsOptions fields for Go config parity" ``` --- ### Task 4: Config Processor **Files:** - Create: `src/NATS.Server/Configuration/ConfigProcessor.cs` - Create: `tests/NATS.Server.Tests/TestData/basic.conf` - Create: `tests/NATS.Server.Tests/TestData/auth.conf` - Create: `tests/NATS.Server.Tests/TestData/tls.conf` - Create: `tests/NATS.Server.Tests/TestData/full.conf` - Test: `tests/NATS.Server.Tests/ConfigProcessorTests.cs` **Step 1: Create test config files** ``` # tests/NATS.Server.Tests/TestData/basic.conf port: 4222 host: "0.0.0.0" server_name: "test-server" max_payload: 2mb max_connections: 1000 debug: true trace: false logtime: true logtime_utc: false ping_interval: "30s" ping_max: 3 write_deadline: "5s" max_subs: 100 max_sub_tokens: 16 max_control_line: 2048 max_pending: 32mb lame_duck_duration: "60s" lame_duck_grace_period: "5s" http_port: 8222 ``` ``` # tests/NATS.Server.Tests/TestData/auth.conf authorization { user: admin password: "s3cret" timeout: 5 users = [ { user: alice, password: "pw1", permissions: { publish: { allow: ["foo.>"] }, subscribe: { allow: [">"] } } } { user: bob, password: "pw2" } ] } no_auth_user: "guest" ``` ``` # tests/NATS.Server.Tests/TestData/tls.conf tls { cert_file: "/path/to/cert.pem" key_file: "/path/to/key.pem" ca_file: "/path/to/ca.pem" verify: true verify_and_map: true timeout: 3 connection_rate_limit: 100 pinned_certs: ["abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789"] handshake_first: true } allow_non_tls: false ``` ``` # tests/NATS.Server.Tests/TestData/full.conf # Full configuration with all supported options port: 4222 host: "0.0.0.0" server_name: "full-test" client_advertise: "nats://public.example.com:4222" max_payload: 1mb max_control_line: 4096 max_connections: 65536 max_pending: 64mb write_deadline: "10s" max_subs: 0 max_sub_tokens: 0 max_traced_msg_len: 1024 disable_sublist_cache: false max_closed_clients: 5000 ping_interval: "2m" ping_max: 2 debug: false trace: false trace_verbose: false logtime: true logtime_utc: false logfile: "/var/log/nats.log" log_size_limit: 100mb log_max_num: 5 http_port: 8222 http_base_path: "/nats" pidfile: "/var/run/nats.pid" ports_file_dir: "/var/run" lame_duck_duration: "2m" lame_duck_grace_period: "10s" server_tags { region: "us-east" env: "production" } authorization { user: admin password: "secret" timeout: 2 } tls { cert_file: "/path/to/cert.pem" key_file: "/path/to/key.pem" ca_file: "/path/to/ca.pem" verify: true timeout: 2 handshake_first: true } ``` **Step 2: Write failing tests** ```csharp // tests/NATS.Server.Tests/ConfigProcessorTests.cs using NATS.Server.Configuration; namespace NATS.Server.Tests; public class ConfigProcessorTests { private static string TestDataPath(string filename) => Path.Combine(AppContext.BaseDirectory, "TestData", filename); [Fact] public void ProcessConfigFile_BasicConf_SetsNetworkOptions() { var opts = ConfigProcessor.ProcessConfigFile(TestDataPath("basic.conf")); opts.Port.ShouldBe(4222); opts.Host.ShouldBe("0.0.0.0"); opts.ServerName.ShouldBe("test-server"); opts.MaxPayload.ShouldBe(2 * 1024 * 1024); opts.MaxConnections.ShouldBe(1000); } [Fact] public void ProcessConfigFile_BasicConf_SetsLoggingOptions() { var opts = ConfigProcessor.ProcessConfigFile(TestDataPath("basic.conf")); opts.Debug.ShouldBeTrue(); opts.Trace.ShouldBeFalse(); opts.Logtime.ShouldBeTrue(); } [Fact] public void ProcessConfigFile_BasicConf_SetsPingOptions() { var opts = ConfigProcessor.ProcessConfigFile(TestDataPath("basic.conf")); opts.PingInterval.ShouldBe(TimeSpan.FromSeconds(30)); opts.MaxPingsOut.ShouldBe(3); } [Fact] public void ProcessConfigFile_BasicConf_SetsLimits() { var opts = ConfigProcessor.ProcessConfigFile(TestDataPath("basic.conf")); opts.WriteDeadline.ShouldBe(TimeSpan.FromSeconds(5)); opts.MaxSubs.ShouldBe(100); opts.MaxSubTokens.ShouldBe(16); opts.MaxControlLine.ShouldBe(2048); opts.MaxPending.ShouldBe(32L * 1024 * 1024); } [Fact] public void ProcessConfigFile_BasicConf_SetsLifecycle() { var opts = ConfigProcessor.ProcessConfigFile(TestDataPath("basic.conf")); opts.LameDuckDuration.ShouldBe(TimeSpan.FromSeconds(60)); opts.LameDuckGracePeriod.ShouldBe(TimeSpan.FromSeconds(5)); } [Fact] public void ProcessConfigFile_BasicConf_SetsMonitoring() { var opts = ConfigProcessor.ProcessConfigFile(TestDataPath("basic.conf")); opts.MonitorPort.ShouldBe(8222); } [Fact] public void ProcessConfigFile_AuthConf_SetsAuthOptions() { var opts = ConfigProcessor.ProcessConfigFile(TestDataPath("auth.conf")); opts.Username.ShouldBe("admin"); opts.Password.ShouldBe("s3cret"); opts.AuthTimeout.ShouldBe(TimeSpan.FromSeconds(5)); opts.NoAuthUser.ShouldBe("guest"); } [Fact] public void ProcessConfigFile_AuthConf_ParsesUsers() { var opts = ConfigProcessor.ProcessConfigFile(TestDataPath("auth.conf")); opts.Users.ShouldNotBeNull(); opts.Users!.Count.ShouldBe(2); opts.Users[0].Username.ShouldBe("alice"); opts.Users[0].Permissions.ShouldNotBeNull(); opts.Users[0].Permissions!.Publish!.Allow!.ShouldContain("foo.>"); opts.Users[1].Username.ShouldBe("bob"); } [Fact] public void ProcessConfigFile_TlsConf_SetsTlsOptions() { var opts = ConfigProcessor.ProcessConfigFile(TestDataPath("tls.conf")); opts.TlsCert.ShouldBe("/path/to/cert.pem"); opts.TlsKey.ShouldBe("/path/to/key.pem"); opts.TlsCaCert.ShouldBe("/path/to/ca.pem"); opts.TlsVerify.ShouldBeTrue(); opts.TlsMap.ShouldBeTrue(); opts.TlsTimeout.ShouldBe(TimeSpan.FromSeconds(3)); opts.TlsRateLimit.ShouldBe(100); opts.TlsPinnedCerts.ShouldNotBeNull(); opts.TlsPinnedCerts!.Count.ShouldBe(1); opts.TlsHandshakeFirst.ShouldBeTrue(); opts.AllowNonTls.ShouldBeFalse(); } [Fact] public void ProcessConfigFile_FullConf_SetsExtendedOptions() { var opts = ConfigProcessor.ProcessConfigFile(TestDataPath("full.conf")); opts.ClientAdvertise.ShouldBe("nats://public.example.com:4222"); opts.MaxTracedMsgLen.ShouldBe(1024); opts.MaxClosedClients.ShouldBe(5000); opts.DisableSublistCache.ShouldBeFalse(); } [Fact] public void ProcessConfigFile_FullConf_ParsesTags() { var opts = ConfigProcessor.ProcessConfigFile(TestDataPath("full.conf")); opts.Tags.ShouldNotBeNull(); opts.Tags!["region"].ShouldBe("us-east"); opts.Tags["env"].ShouldBe("production"); } [Fact] public void ProcessConfigFile_ListenCombined_SetsHostAndPort() { var opts = ConfigProcessor.ProcessConfig("listen: \"127.0.0.1:5222\""); opts.Host.ShouldBe("127.0.0.1"); opts.Port.ShouldBe(5222); } [Fact] public void ProcessConfigFile_HttpCombined_SetsMonitorHostAndPort() { var opts = ConfigProcessor.ProcessConfig("http: \"0.0.0.0:9222\""); opts.MonitorHost.ShouldBe("0.0.0.0"); opts.MonitorPort.ShouldBe(9222); } [Fact] public void ProcessConfigFile_DurationAsNumber_TreatedAsSeconds() { // Backward compat: bare number = seconds var opts = ConfigProcessor.ProcessConfig("ping_interval: 30"); opts.PingInterval.ShouldBe(TimeSpan.FromSeconds(30)); } [Fact] public void ProcessConfigFile_UnknownKeys_SilentlyIgnored() { // Cluster/JetStream keys should not cause errors var opts = ConfigProcessor.ProcessConfig("port: 4222\ncluster { port: 6222 }\njetstream: true"); opts.Port.ShouldBe(4222); } [Fact] public void ProcessConfigFile_ServerNameWithSpaces_ThrowsError() { Should.Throw(() => ConfigProcessor.ProcessConfig("server_name: \"has spaces\"")); } [Fact] public void ProcessConfigFile_MaxSubTokensTooLarge_ThrowsError() { Should.Throw(() => ConfigProcessor.ProcessConfig("max_sub_tokens: 300")); } } ``` **Step 2b: Ensure TestData files are copied to output** Add to `tests/NATS.Server.Tests/NATS.Server.Tests.csproj`: ```xml ``` **Step 3: Run tests to verify they fail** Run: `dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~ConfigProcessorTests" -v normal` Expected: FAIL — `ConfigProcessor` doesn't exist **Step 4: Implement the config processor** Create `src/NATS.Server/Configuration/ConfigProcessor.cs`. Port the option processing from Go `opts.go:1050-1400` (the `processConfigFileLine` switch). Key methods: ```csharp public static class ConfigProcessor { public static NatsOptions ProcessConfigFile(string filePath) { var config = NatsConfParser.ParseFile(filePath); var opts = new NatsOptions { ConfigFile = filePath }; ApplyConfig(config, opts); return opts; } public static NatsOptions ProcessConfig(string configText) { var config = NatsConfParser.Parse(configText); var opts = new NatsOptions(); ApplyConfig(config, opts); return opts; } public static void ApplyConfig(Dictionary config, NatsOptions opts) { ... } } ``` `ApplyConfig` iterates the dictionary, switching on `key.ToLowerInvariant()`: - `"listen"` → `ParseListen(value)` to split `"host:port"` into `opts.Host`/`opts.Port` - `"port"` → `opts.Port = (int)(long)value` - `"host"`, `"net"` → `opts.Host = (string)value` - `"server_name"` → validate no spaces, set `opts.ServerName` - `"debug"` → `opts.Debug = (bool)value` - `"trace"` → `opts.Trace = (bool)value` - `"trace_verbose"` → `opts.TraceVerbose = (bool)value; opts.Trace = (bool)value` - `"authorization"` → `ParseAuthorization((Dictionary)value, opts)` - `"tls"` → `ParseTls((Dictionary)value, opts)` - `"server_tags"` → `ParseTags((Dictionary)value, opts)` - Duration fields (`ping_interval`, `write_deadline`, `lame_duck_duration`, `lame_duck_grace_period`) → `ParseDuration()` helper - Unknown keys → silently ignored (for cluster/JetStream forward compat) Helper methods: - `ParseListen(object value)` → splits `"host:port"` string - `ParseDuration(string field, object value)` → handles string durations (`"30s"`, `"2m"`) and bare numbers (seconds) - `ParseAuthorization(dict, opts)` → processes `user`, `password`, `token`, `timeout`, `users` array - `ParseUsers(List users)` → builds `List` with optional permissions - `ParsePermissions(dict)` → builds `Permissions` with `SubjectPermission` (allow/deny lists) - `ParseTls(dict, opts)` → maps TLS block keys to TLS options - `ParseTags(dict, opts)` → builds `Dictionary` tags Also create `ConfigProcessorException` for error reporting: ```csharp public sealed class ConfigProcessorException(string message, List errors) : Exception(message) { public IReadOnlyList Errors => errors; } ``` Errors are collected into a `List` during processing. If any errors exist after processing all keys, throw `ConfigProcessorException` with all errors. **Step 5: Run tests to verify they pass** Run: `dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~ConfigProcessorTests" -v normal` Expected: PASS **Step 6: Commit** ```bash git add src/NATS.Server/Configuration/ConfigProcessor.cs tests/NATS.Server.Tests/ConfigProcessorTests.cs tests/NATS.Server.Tests/TestData/ tests/NATS.Server.Tests/NATS.Server.Tests.csproj git commit -m "feat: add config processor mapping parsed config to NatsOptions" ``` --- ### Task 5: Hot Reload System **Files:** - Create: `src/NATS.Server/Configuration/IConfigChange.cs` - Create: `src/NATS.Server/Configuration/ConfigReloader.cs` - Modify: `src/NATS.Server/NatsServer.cs` (SIGHUP handler + `ReloadConfig()`) - Test: `tests/NATS.Server.Tests/ConfigReloadTests.cs` **Step 1: Write failing tests** ```csharp // tests/NATS.Server.Tests/ConfigReloadTests.cs using NATS.Server.Configuration; namespace NATS.Server.Tests; public class ConfigReloadTests { [Fact] public void Diff_NoChanges_ReturnsEmpty() { var old = new NatsOptions { Port = 4222, Debug = true }; var @new = new NatsOptions { Port = 4222, Debug = true }; var changes = ConfigReloader.Diff(old, @new); changes.ShouldBeEmpty(); } [Fact] public void Diff_ReloadableChange_ReturnsChange() { var old = new NatsOptions { Debug = false }; var @new = new NatsOptions { Debug = true }; var changes = ConfigReloader.Diff(old, @new); changes.Count.ShouldBe(1); changes[0].Name.ShouldBe("Debug"); changes[0].IsLoggingChange.ShouldBeTrue(); } [Fact] public void Diff_NonReloadableChange_ReturnsNonReloadableChange() { var old = new NatsOptions { Port = 4222 }; var @new = new NatsOptions { Port = 5222 }; var changes = ConfigReloader.Diff(old, @new); changes.Count.ShouldBe(1); changes[0].IsNonReloadable.ShouldBeTrue(); } [Fact] public void Diff_MultipleChanges_ReturnsAll() { var old = new NatsOptions { Debug = false, MaxPayload = 1024 }; var @new = new NatsOptions { Debug = true, MaxPayload = 2048 }; var changes = ConfigReloader.Diff(old, @new); changes.Count.ShouldBe(2); } [Fact] public void Diff_AuthChange_MarkedCorrectly() { var old = new NatsOptions { Username = "alice" }; var @new = new NatsOptions { Username = "bob" }; var changes = ConfigReloader.Diff(old, @new); changes[0].IsAuthChange.ShouldBeTrue(); } [Fact] public void Diff_TlsChange_MarkedCorrectly() { var old = new NatsOptions { TlsCert = "/old/cert.pem" }; var @new = new NatsOptions { TlsCert = "/new/cert.pem" }; var changes = ConfigReloader.Diff(old, @new); changes[0].IsTlsChange.ShouldBeTrue(); } [Fact] public void Validate_NonReloadableChanges_ReturnsErrors() { var changes = new List { new ConfigChange("Port", isNonReloadable: true), }; var errors = ConfigReloader.Validate(changes); errors.Count.ShouldBe(1); errors[0].ShouldContain("Port"); } [Fact] public void MergeWithCli_CliOverridesConfig() { var fromConfig = new NatsOptions { Port = 5222, Debug = true }; var cliFlags = new HashSet { "Port" }; var cliValues = new NatsOptions { Port = 4222 }; ConfigReloader.MergeCliOverrides(fromConfig, cliValues, cliFlags); fromConfig.Port.ShouldBe(4222); // CLI wins fromConfig.Debug.ShouldBeTrue(); // config value kept (not in CLI) } } ``` **Step 2: Run tests to verify they fail** Run: `dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~ConfigReloadTests" -v normal` Expected: FAIL **Step 3: Implement the change interface and reloader** ```csharp // src/NATS.Server/Configuration/IConfigChange.cs namespace NATS.Server.Configuration; public interface IConfigChange { string Name { get; } bool IsLoggingChange { get; } bool IsAuthChange { get; } bool IsTlsChange { get; } bool IsNonReloadable { get; } } public sealed class ConfigChange( string name, bool isLoggingChange = false, bool isAuthChange = false, bool isTlsChange = false, bool isNonReloadable = false) : IConfigChange { public string Name => name; public bool IsLoggingChange => isLoggingChange; public bool IsAuthChange => isAuthChange; public bool IsTlsChange => isTlsChange; public bool IsNonReloadable => isNonReloadable; } ``` ```csharp // src/NATS.Server/Configuration/ConfigReloader.cs namespace NATS.Server.Configuration; public static class ConfigReloader { // Non-reloadable options (match Go server) private static readonly HashSet NonReloadable = ["Host", "Port", "ServerName"]; // Logging-related options private static readonly HashSet LoggingOptions = ["Debug", "Trace", "TraceVerbose", "Logtime", "LogtimeUTC", "LogFile", "LogSizeLimit", "LogMaxFiles", "Syslog", "RemoteSyslog"]; // Auth-related options private static readonly HashSet AuthOptions = ["Username", "Password", "Authorization", "Users", "NKeys", "NoAuthUser", "AuthTimeout"]; // TLS-related options private static readonly HashSet TlsOptions = ["TlsCert", "TlsKey", "TlsCaCert", "TlsVerify", "TlsMap", "TlsTimeout", "TlsHandshakeFirst", "TlsHandshakeFirstFallback", "AllowNonTls", "TlsRateLimit", "TlsPinnedCerts"]; public static List Diff(NatsOptions oldOpts, NatsOptions newOpts) { ... } public static List Validate(List changes) { ... } public static void MergeCliOverrides(NatsOptions fromConfig, NatsOptions cliValues, HashSet cliFlags) { ... } } ``` `Diff` compares each property of `NatsOptions` using reflection or explicit property-by-property checks. For each changed property, creates a `ConfigChange` with the appropriate flags. `Validate` returns error messages for any `IsNonReloadable` changes. `MergeCliOverrides` copies properties listed in `cliFlags` from `cliValues` back to `fromConfig`. **Step 4: Run tests to verify they pass** Run: `dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~ConfigReloadTests" -v normal` Expected: PASS **Step 5: Commit** ```bash git add src/NATS.Server/Configuration/IConfigChange.cs src/NATS.Server/Configuration/ConfigReloader.cs tests/NATS.Server.Tests/ConfigReloadTests.cs git commit -m "feat: add config reloader with diff, validate, and CLI merge" ``` --- ### Task 6: Server Integration (Config Loading + SIGHUP Reload) **Files:** - Modify: `src/NATS.Server/NatsServer.cs` (constructor, SIGHUP handler, new `ReloadConfig()`) - Modify: `src/NATS.Server.Host/Program.cs` (config file loading, CLI tracking) - Test: `tests/NATS.Server.Tests/ConfigIntegrationTests.cs` (new file) **Step 1: Write failing integration tests** ```csharp // tests/NATS.Server.Tests/ConfigIntegrationTests.cs using NATS.Server.Configuration; namespace NATS.Server.Tests; public class ConfigIntegrationTests { [Fact] public void Server_WithConfigFile_LoadsOptionsFromFile() { var dir = Path.Combine(Path.GetTempPath(), $"nats_test_{Guid.NewGuid():N}"); Directory.CreateDirectory(dir); try { var confPath = Path.Combine(dir, "test.conf"); File.WriteAllText(confPath, "port: 14222\nmax_payload: 2mb\ndebug: true"); var opts = ConfigProcessor.ProcessConfigFile(confPath); opts.Port.ShouldBe(14222); opts.MaxPayload.ShouldBe(2 * 1024 * 1024); opts.Debug.ShouldBeTrue(); } finally { Directory.Delete(dir, true); } } [Fact] public void Server_CliOverridesConfig() { var dir = Path.Combine(Path.GetTempPath(), $"nats_test_{Guid.NewGuid():N}"); Directory.CreateDirectory(dir); try { var confPath = Path.Combine(dir, "test.conf"); File.WriteAllText(confPath, "port: 14222\ndebug: true"); var opts = ConfigProcessor.ProcessConfigFile(confPath); // Simulate CLI override opts.InCmdLine.Add("Port"); var cliOpts = new NatsOptions { Port = 5222 }; ConfigReloader.MergeCliOverrides(opts, cliOpts, opts.InCmdLine); opts.Port.ShouldBe(5222); // CLI wins opts.Debug.ShouldBeTrue(); // config value preserved } finally { Directory.Delete(dir, true); } } [Fact] public void Reload_ChangingPort_ReturnsError() { var oldOpts = new NatsOptions { Port = 4222 }; var newOpts = new NatsOptions { Port = 5222 }; var changes = ConfigReloader.Diff(oldOpts, newOpts); var errors = ConfigReloader.Validate(changes); errors.Count.ShouldBeGreaterThan(0); errors[0].ShouldContain("Port"); } [Fact] public void Reload_ChangingDebug_IsValid() { var oldOpts = new NatsOptions { Debug = false }; var newOpts = new NatsOptions { Debug = true }; var changes = ConfigReloader.Diff(oldOpts, newOpts); var errors = ConfigReloader.Validate(changes); errors.ShouldBeEmpty(); changes.ShouldContain(c => c.IsLoggingChange); } } ``` **Step 2: Run tests to verify they fail (or pass if processor already works)** Run: `dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~ConfigIntegrationTests" -v normal` **Step 3: Update NatsServer.cs** Add `ReloadConfig()` method and update SIGHUP handler. In the constructor (around line 248), after storing options: ```csharp // Store a snapshot of CLI-set flags for reload precedence _cliSnapshot = new NatsOptions { /* copy current values */ }; _cliFlags = options.InCmdLine; ``` Add new fields: ```csharp private readonly NatsOptions? _cliSnapshot; private readonly HashSet _cliFlags; private string? _configDigest; ``` Update SIGHUP handler (line 223-227): ```csharp _signalRegistrations.Add(PosixSignalRegistration.Create(PosixSignal.SIGHUP, ctx => { ctx.Cancel = true; _logger.LogInformation("Trapped SIGHUP signal — reloading configuration"); _ = Task.Run(() => ReloadConfig()); })); ``` Add `ReloadConfig()`: ```csharp public void ReloadConfig() { if (_options.ConfigFile == null) { _logger.LogWarning("No config file specified, cannot reload"); return; } try { var (newConfig, digest) = NatsConfParser.ParseFileWithDigest(_options.ConfigFile); if (digest == _configDigest) { _logger.LogInformation("Config file unchanged, no reload needed"); return; } var newOpts = new NatsOptions { ConfigFile = _options.ConfigFile }; ConfigProcessor.ApplyConfig(newConfig, newOpts); // CLI flags override config if (_cliSnapshot != null) ConfigReloader.MergeCliOverrides(newOpts, _cliSnapshot, _cliFlags); var changes = ConfigReloader.Diff(_options, newOpts); var errors = ConfigReloader.Validate(changes); if (errors.Count > 0) { foreach (var err in errors) _logger.LogError("Config reload error: {Error}", err); return; } // Apply changes ApplyConfigChanges(changes, newOpts); _configDigest = digest; _logger.LogInformation("Config reloaded successfully ({Count} changes applied)", changes.Count); } catch (Exception ex) { _logger.LogError(ex, "Failed to reload config file: {ConfigFile}", _options.ConfigFile); } } ``` Add `ApplyConfigChanges()` which copies new values to `_options` and triggers side effects: - If any `IsLoggingChange` → invoke `ReOpenLogFile` callback - If any `IsAuthChange` → rebuild `_authService` - If any `IsTlsChange` → rebuild `_sslOptions` (log warning, actual TLS reload requires cert reload hook) - Update limits directly on `_options` **Step 4: Update Program.cs** Before CLI parsing, add config file loading: ```csharp // After extracting -c flag but before other CLI args string? configFile = null; for (int i = 0; i < args.Length; i++) { if (args[i] == "-c" && i + 1 < args.Length) { configFile = args[i + 1]; break; } } if (configFile != null) { options = ConfigProcessor.ProcessConfigFile(configFile); options.ConfigFile = configFile; } // Then apply CLI args (overriding config values) and track in InCmdLine ``` Update the CLI switch to track overrides: ```csharp case "-p" or "--port" when i + 1 < args.Length: options.Port = int.Parse(args[++i]); options.InCmdLine.Add("Port"); break; // ... same pattern for all other CLI flags ``` Remove the startup warning about config file not being supported: ```csharp // DELETE these lines from NatsServer.cs StartAsync: // if (_options.ConfigFile != null) // _logger.LogWarning("Config file parsing not yet supported..."); ``` **Step 5: Run all tests** Run: `dotnet test tests/NATS.Server.Tests -v normal` Expected: PASS **Step 6: Commit** ```bash git add src/NATS.Server/NatsServer.cs src/NATS.Server.Host/Program.cs tests/NATS.Server.Tests/ConfigIntegrationTests.cs git commit -m "feat: integrate config file loading and SIGHUP hot reload" ``` --- ### Task 7: Update differences.md **Files:** - Modify: `differences.md` **Step 1: Update the config parsing entries** In section 6 (Configuration), update: - `Config file parsing | Y | N` → `Config file parsing | Y | Y | Custom NATS conf parser ported from Go` - `Hot reload (SIGHUP) | Y | N` → `Hot reload (SIGHUP) | Y | Y | Reloads logging, auth, limits, TLS certs on SIGHUP` - `Config change detection | Y | N` → `Config change detection | Y | Y | SHA256 digest comparison; CLI flag precedence tracking` - Update `~62` options count to reflect new fields In section 1 (Core Server Lifecycle): - `Config file validation on startup | Y | Stub` → `Config file validation on startup | Y | Y | Full config parsing with error collection` In section 1 (Signal Handling): - `SIGHUP (config reload) | Y | Stub` → `SIGHUP (config reload) | Y | Y | Re-parses config, diffs options, applies reloadable subset` In Summary section: - Move config file parsing and hot reload from "Remaining High Priority" to "Resolved Since Initial Audit" - Remove the "Remaining High Priority" section entirely if empty **Step 2: Verify the document is accurate** Read through the full differences.md to ensure all updated entries match actual implementation. **Step 3: Commit** ```bash git add differences.md git commit -m "docs: update differences.md to reflect config parsing and hot reload implementation" ``` --- ### Task 8: Full Test Suite Verification **Step 1: Run the full test suite** Run: `dotnet test tests/NATS.Server.Tests -v normal` Expected: ALL PASS **Step 2: Build the solution** Run: `dotnet build` Expected: Build succeeded with 0 errors **Step 3: Smoke test with config file** Create a temp config file and verify the server starts with it: ```bash echo 'port: 14222\ndebug: true\nmax_payload: 2mb' > /tmp/test-nats.conf dotnet run --project src/NATS.Server.Host -- -c /tmp/test-nats.conf & sleep 2 curl -s http://localhost:8222/varz | grep -q "14222" # verify port from config kill %1 ``` **Step 4: Final commit (if any fixes needed)** ```bash git add -A git commit -m "fix: address issues found during full test verification" ```