diff --git a/CLAUDE.md b/CLAUDE.md index d7409f1..80a6078 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -124,7 +124,7 @@ powershell -ExecutionPolicy Bypass -File scripts/run-client-e2e-tests.ps1 When source code changes, build and test the affected component before reporting work done. If the change crosses component boundaries, build each affected component — don't rely on a single top-level build: -**Run targeted tests per task, never the full suite each time.** When executing a plan task-by-task, run only the tests that exercise the code that task touched (`dotnet test --filter "FullyQualifiedName~"`, or the per-task test named in the plan). The full gateway suite is slow — run it at most once per phase (after a related batch of tasks lands), not after every task. This is a speed guideline, not a correctness one: the suite exits cleanly (verified on macOS and the Windows dev box — 0 surviving `testhost`/worker processes after a full run), so filtered runs are about turnaround, not about avoiding a process leak. +**Run targeted tests per task, never the full suite each time.** When executing a plan task-by-task, run only the tests that exercise the code that task touched (`dotnet test --filter "FullyQualifiedName~"`, or the per-task test named in the plan). The full gateway suite is slow — run it at most once per phase (after a related batch of tasks lands), not after every task. This is a speed guideline on macOS, where the suite exits cleanly (0 surviving `testhost`/worker processes after a full run) — filtered runs there are about turnaround, not about avoiding a process leak. **On windev a full-suite run wedges**: every test completes, then the x64 `testhost` never exits and `dotnet test` never returns (filtered runs exit normally). Run the full suite there with `--blame-hang --blame-hang-timeout 5m --blame-hang-dump-type none` and read the summary line, not the exit code — see "Running the Gateway Suite on windev" in `docs/GatewayTesting.md`. | Changed area | Required verification | |---|---| diff --git a/docs/GatewayTesting.md b/docs/GatewayTesting.md index fc75981..df8a0d3 100644 --- a/docs/GatewayTesting.md +++ b/docs/GatewayTesting.md @@ -485,6 +485,88 @@ Run the gateway test project after shared gateway test infrastructure changes: dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj ``` +## Running the Gateway Suite on windev + +The gateway suite (`ZB.MOM.WW.MxGateway.Tests`, net10.0/x64) is not part of the CI +Windows tier — `windows-x86` and `nightly-windev` run only the x86 Worker build and +`Worker.Tests`. It is still run on windev by hand when a change needs Windows +confirmation, and that run has three Windows-specific characteristics worth knowing +before results are interpreted. + +Run it from an isolated clone under `C:\build` checked out to the SHA under test — never +the dirty Desktop checkout, and never the CI clone `C:\build\mxaccessgw-ci`, whose worktree +lock belongs to the Worker tier. + +Baseline on an otherwise idle windev (2026-08-10): **855 passed, 0 failed, 29 s**. The +suite is smaller there than the 879 the macOS box runs because some cases are gated to +Unix. Any failure is therefore a real signal — but read the load caveat below before +acting on one. + +### Two long-standing "windev-environmental" failures were test bugs, not the environment + +Both were dismissed as environmental for months and are now fixed. Neither depended on +anything installed on windev; both failed on **any** Windows host: + +- `SelfSignedCertificateProviderTests.GenerateCertificate_HasExpectedSansEkuAndValidity` + asserted SAN content by substring-matching `X509Extension.Format(false)`. That string is + produced by the platform crypto library: Windows' `CryptFormatObject` renders the IPv6 + loopback fully expanded (`IP Address=0000:0000:0000:0000:0000:0000:0000:0001`) while the + managed formatter used on macOS/Linux renders `::1`, so the loopback assertion failed on + Windows only. The test now decodes the extension with `X509SubjectAlternativeNameExtension` + and compares parsed `IPAddress` values and DNS names (case-insensitively, as DNS names + are), which is platform-independent. +- `SessionManagerTests.OpenSessionAsync_PipeNameIsShortAndUniquePerPidAndSession` guards the + 104-byte macOS `sun_path` budget that NEXT-01 shortened the pipe name to fit. It padded the + measured name up to a five-digit pid but never substituted that worst case *downward*, so a + six-digit pid — routine on Windows, impossible on macOS, where pids stop at 99999 — made the + name one character "too long" against a budget that does not apply to the host running the + test. The check now replaces the running pid's digit count with the five-digit macOS worst + case, so it measures the name *format* rather than the current process's pid. + +### The real-pipe suites are load-sensitive + +These suites drive real named pipes against a five-second worker startup timeout and start +failing when windev is busy — most often when the x86 Worker tier is building or testing at +the same time. All five passed in the idle baseline above and all five failed in a run taken +while an x86 build and `Worker.Tests` were in flight (that run also took 2 m 21 s against the +idle 29 s): + +- `GatewayEndToEndFakeWorkerSmokeTests`, `GatewayEndToEndMultiSubscriberTests`, + `GatewayEndToEndReconnectReplayTests` — fail as + `RpcException Status(StatusCode="Unavailable", Detail="Failed to open session …")`. +- `SessionWorkerClientFactoryFakeWorkerTests.CreateAsync_WhenFakeWorkerStartupFails_ThrowsWorkerClientException` + — the startup timeout beats the protocol violation the test is asserting, so the observed + exception is `TimeoutException` instead of `WorkerClientException`. +- `WorkerClientTests.InvokeAsync_WhenCommandExceedsFrameMax_FailsOnlyThatCommandAndStaysReady`. +- `EventStreamServiceTests.StreamEventsAsync_WithConcurrentStreams_TracksAggregateQueueDepth` + — polls a metric against a five-second deadline. Its helper now reports the unmet condition + rather than letting a bare `TaskCanceledException` escape, so a load-induced timeout here + names what it was waiting for instead of looking like an unexplained cancellation. + +A failure in that list is evidence about machine load, not about the change under test. Check +for a concurrent x86 build/test (`Get-Process dotnet, testhost, testhost.net48.x86, +MSBuild, VBCSCompiler`) and re-run the affected class on its own before treating it as real. +windev has 36 logical CPUs and `xunit.runner.json` sets `maxParallelThreads: -1`, so the +suite runs far wider there than on the macOS dev box — that width is what turns these +real-clock deadlines into failures. + +### The full-suite testhost does not exit on windev + +After the last test completes, the x64 `testhost` process stops doing work but never exits, +so `dotnet test` never returns and the run has to be killed. This does **not** happen on +filtered runs (`--filter …`), which exit normally, and does not happen on the macOS dev box — +it is specific to a full-suite run on windev. Run the full suite with a hang guard so the +wedged host is torn down and the pass/fail summary is still printed: + +```powershell +dotnet test src\ZB.MOM.WW.MxGateway.Tests\ZB.MOM.WW.MxGateway.Tests.csproj ` + --blame-hang --blame-hang-timeout 5m --blame-hang-dump-type none +``` + +The summary line ahead of the abort is the real result; the process exit code is nonzero +because of the abort even when every test passed, so read the summary rather than the exit +code. Prefer filtered runs on windev whenever the change under test allows it. + ## Continuous Integration CI runs on Gitea Actions (`.gitea/workflows/ci.yml`; origin is Gitea at diff --git a/src/ZB.MOM.WW.MxGateway.Tests/Gateway/Grpc/EventStreamServiceTests.cs b/src/ZB.MOM.WW.MxGateway.Tests/Gateway/Grpc/EventStreamServiceTests.cs index f871fe9..5695f44 100644 --- a/src/ZB.MOM.WW.MxGateway.Tests/Gateway/Grpc/EventStreamServiceTests.cs +++ b/src/ZB.MOM.WW.MxGateway.Tests/Gateway/Grpc/EventStreamServiceTests.cs @@ -699,12 +699,26 @@ public sealed class EventStreamServiceTests }; } - private static async Task WaitUntilAsync(Func predicate) + // The real-clock deadline here is load-sensitive on a wide host (windev runs this suite + // 36-way parallel). Surfacing the unmet condition instead of letting the bare + // TaskCanceledException escape is what makes such a failure diagnosable rather than a + // mystery cancellation attributed to "the environment". + private static async Task WaitUntilAsync( + Func predicate, + [CallerArgumentExpression(nameof(predicate))] string? predicateExpression = null) { using CancellationTokenSource cancellationTokenSource = new(TestTimeout); while (!predicate()) { - await Task.Delay(TimeSpan.FromMilliseconds(10), cancellationTokenSource.Token); + try + { + await Task.Delay(TimeSpan.FromMilliseconds(10), cancellationTokenSource.Token); + } + catch (OperationCanceledException) + { + Assert.Fail( + $"Timed out after {TestTimeout} waiting for condition: {predicateExpression}"); + } } } diff --git a/src/ZB.MOM.WW.MxGateway.Tests/Gateway/Sessions/SessionManagerTests.cs b/src/ZB.MOM.WW.MxGateway.Tests/Gateway/Sessions/SessionManagerTests.cs index 397de15..c320ff1 100644 --- a/src/ZB.MOM.WW.MxGateway.Tests/Gateway/Sessions/SessionManagerTests.cs +++ b/src/ZB.MOM.WW.MxGateway.Tests/Gateway/Sessions/SessionManagerTests.cs @@ -61,10 +61,20 @@ public sealed class SessionManagerTests // 104-byte sun_path − NUL − ~49-char default macOS TMPDIR − "CoreFxPipe_". const int MaxPipeNameLength = 104 - 1 - 49 - 11; - int worstCasePidDigits = 5 - Environment.ProcessId.ToString(System.Globalization.CultureInfo.InvariantCulture).Length; + + // macOS pids top out at 99999, so 5 digits is the worst case the budget must survive. + // The check must substitute that worst case for the *running* pid's digit count rather + // than pad the measured length upward: Windows pids are routinely 6 digits, which would + // otherwise fail this assertion on a host whose own pipe-name limit (256) is irrelevant + // to the macOS budget being guarded here. + const int WorstCaseMacOsPidDigits = 5; + int runningPidDigits = Environment.ProcessId + .ToString(System.Globalization.CultureInfo.InvariantCulture).Length; + int worstCaseLength = session.PipeName.Length - runningPidDigits + WorstCaseMacOsPidDigits; Assert.True( - session.PipeName.Length + Math.Max(0, worstCasePidDigits) <= MaxPipeNameLength, - $"Pipe name '{session.PipeName}' would overflow the macOS socket-path budget at a 5-digit pid."); + worstCaseLength <= MaxPipeNameLength, + $"Pipe name '{session.PipeName}' would overflow the macOS socket-path budget at a 5-digit pid " + + $"({worstCaseLength} > {MaxPipeNameLength})."); } /// Verifies that a session opened by an authenticated caller records that caller's API key id in OwnerKeyId. diff --git a/src/ZB.MOM.WW.MxGateway.Tests/Security/Tls/SelfSignedCertificateProviderTests.cs b/src/ZB.MOM.WW.MxGateway.Tests/Security/Tls/SelfSignedCertificateProviderTests.cs index 1a4cd69..6b1554e 100644 --- a/src/ZB.MOM.WW.MxGateway.Tests/Security/Tls/SelfSignedCertificateProviderTests.cs +++ b/src/ZB.MOM.WW.MxGateway.Tests/Security/Tls/SelfSignedCertificateProviderTests.cs @@ -1,3 +1,4 @@ +using System.Net; using System.Security.Cryptography.X509Certificates; using Microsoft.Extensions.Logging.Abstractions; using Microsoft.Extensions.Time.Testing; @@ -25,15 +26,19 @@ public sealed class SelfSignedCertificateProviderTests Assert.True(cert.NotBefore.ToUniversalTime() < time.GetUtcNow().UtcDateTime); Assert.True(cert.HasPrivateKey); - string sans = ReadSubjectAltNames(cert); - Assert.Contains("localhost", sans); - Assert.Contains("gw.internal", sans); - Assert.Contains(Environment.MachineName, sans); - // Format() renders IP SANs as "IP Address:"; the IPv6 loopback may appear - // as "::1" or its expanded form depending on the platform crypto library. - Assert.Contains("127.0.0.1", sans); - Assert.True(sans.Contains("::1") || sans.Contains("0:0:0:0:0:0:0:1"), - $"Expected IPv6 loopback in SANs but got: {sans}"); + X509SubjectAlternativeNameExtension san = ReadSubjectAltNames(cert); + string[] dnsNames = [.. san.EnumerateDnsNames()]; + IPAddress[] ipAddresses = [.. san.EnumerateIPAddresses()]; + + // DNS SANs are compared case-insensitively (DNS names are), and IP SANs are compared + // as parsed IPAddress values. Asserting against the extension's Format() string instead + // would be platform-dependent: Windows' CryptFormatObject renders the IPv6 loopback + // fully expanded ("0000:0000:...:0001") while the managed formatter renders "::1". + Assert.Contains(dnsNames, name => name.Equals("localhost", StringComparison.OrdinalIgnoreCase)); + Assert.Contains(dnsNames, name => name.Equals("gw.internal", StringComparison.OrdinalIgnoreCase)); + Assert.Contains(dnsNames, name => name.Equals(Environment.MachineName, StringComparison.OrdinalIgnoreCase)); + Assert.Contains(IPAddress.Loopback, ipAddresses); + Assert.Contains(IPAddress.IPv6Loopback, ipAddresses); X509EnhancedKeyUsageExtension eku = cert.Extensions.OfType().Single(); Assert.Contains(eku.EnhancedKeyUsages.Cast(), @@ -155,8 +160,6 @@ public sealed class SelfSignedCertificateProviderTests private const string SubjectAltNameOid = "2.5.29.17"; - private static string ReadSubjectAltNames(X509Certificate2 cert) - => cert.Extensions - .First(e => e.Oid?.Value == SubjectAltNameOid) - .Format(false); + private static X509SubjectAlternativeNameExtension ReadSubjectAltNames(X509Certificate2 cert) + => new(cert.Extensions.First(e => e.Oid?.Value == SubjectAltNameOid).RawData); }