Merge fix/windev-baseline-env-failures: the two 'environmental' windev failures were Windows-only test bugs

CLAUDE.md

@@ -124,7 +124,7 @@ powershell -ExecutionPolicy Bypass -File scripts/run-client-e2e-tests.ps1
 
 When source code changes, build and test the affected component before reporting work done. If the change crosses component boundaries, build each affected component — don't rely on a single top-level build:
 
-**Run targeted tests per task, never the full suite each time.** When executing a plan task-by-task, run only the tests that exercise the code that task touched (`dotnet test --filter "FullyQualifiedName~<TestClass>"`, or the per-task test named in the plan). The full gateway suite is slow — run it at most once per phase (after a related batch of tasks lands), not after every task. This is a speed guideline, not a correctness one: the suite exits cleanly (verified on macOS and the Windows dev box — 0 surviving `testhost`/worker processes after a full run), so filtered runs are about turnaround, not about avoiding a process leak.
+**Run targeted tests per task, never the full suite each time.** When executing a plan task-by-task, run only the tests that exercise the code that task touched (`dotnet test --filter "FullyQualifiedName~<TestClass>"`, or the per-task test named in the plan). The full gateway suite is slow — run it at most once per phase (after a related batch of tasks lands), not after every task. This is a speed guideline on macOS, where the suite exits cleanly (0 surviving `testhost`/worker processes after a full run) — filtered runs there are about turnaround, not about avoiding a process leak. **On windev a full-suite run wedges**: every test completes, then the x64 `testhost` never exits and `dotnet test` never returns (filtered runs exit normally). Run the full suite there with `--blame-hang --blame-hang-timeout 5m --blame-hang-dump-type none` and read the summary line, not the exit code — see "Running the Gateway Suite on windev" in `docs/GatewayTesting.md`.
 
 | Changed area | Required verification |
 |---|---|

docs/GatewayTesting.md

@@ -485,6 +485,88 @@ Run the gateway test project after shared gateway test infrastructure changes:
 dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj
 ```
 
+## Running the Gateway Suite on windev
+
+The gateway suite (`ZB.MOM.WW.MxGateway.Tests`, net10.0/x64) is not part of the CI
+Windows tier — `windows-x86` and `nightly-windev` run only the x86 Worker build and
+`Worker.Tests`. It is still run on windev by hand when a change needs Windows
+confirmation, and that run has three Windows-specific characteristics worth knowing
+before results are interpreted.
+
+Run it from an isolated clone under `C:\build` checked out to the SHA under test — never
+the dirty Desktop checkout, and never the CI clone `C:\build\mxaccessgw-ci`, whose worktree
+lock belongs to the Worker tier.
+
+Baseline on an otherwise idle windev (2026-08-10): **855 passed, 0 failed, 29 s**. The
+suite is smaller there than the 879 the macOS box runs because some cases are gated to
+Unix. Any failure is therefore a real signal — but read the load caveat below before
+acting on one.
+
+### Two long-standing "windev-environmental" failures were test bugs, not the environment
+
+Both were dismissed as environmental for months and are now fixed. Neither depended on
+anything installed on windev; both failed on **any** Windows host:
+
+- `SelfSignedCertificateProviderTests.GenerateCertificate_HasExpectedSansEkuAndValidity`
+  asserted SAN content by substring-matching `X509Extension.Format(false)`. That string is
+  produced by the platform crypto library: Windows' `CryptFormatObject` renders the IPv6
+  loopback fully expanded (`IP Address=0000:0000:0000:0000:0000:0000:0000:0001`) while the
+  managed formatter used on macOS/Linux renders `::1`, so the loopback assertion failed on
+  Windows only. The test now decodes the extension with `X509SubjectAlternativeNameExtension`
+  and compares parsed `IPAddress` values and DNS names (case-insensitively, as DNS names
+  are), which is platform-independent.
+- `SessionManagerTests.OpenSessionAsync_PipeNameIsShortAndUniquePerPidAndSession` guards the
+  104-byte macOS `sun_path` budget that NEXT-01 shortened the pipe name to fit. It padded the
+  measured name up to a five-digit pid but never substituted that worst case *downward*, so a
+  six-digit pid — routine on Windows, impossible on macOS, where pids stop at 99999 — made the
+  name one character "too long" against a budget that does not apply to the host running the
+  test. The check now replaces the running pid's digit count with the five-digit macOS worst
+  case, so it measures the name *format* rather than the current process's pid.
+
+### The real-pipe suites are load-sensitive
+
+These suites drive real named pipes against a five-second worker startup timeout and start
+failing when windev is busy — most often when the x86 Worker tier is building or testing at
+the same time. All five passed in the idle baseline above and all five failed in a run taken
+while an x86 build and `Worker.Tests` were in flight (that run also took 2 m 21 s against the
+idle 29 s):
+
+- `GatewayEndToEndFakeWorkerSmokeTests`, `GatewayEndToEndMultiSubscriberTests`,
+  `GatewayEndToEndReconnectReplayTests` — fail as
+  `RpcException Status(StatusCode="Unavailable", Detail="Failed to open session …")`.
+- `SessionWorkerClientFactoryFakeWorkerTests.CreateAsync_WhenFakeWorkerStartupFails_ThrowsWorkerClientException`
+  — the startup timeout beats the protocol violation the test is asserting, so the observed
+  exception is `TimeoutException` instead of `WorkerClientException`.
+- `WorkerClientTests.InvokeAsync_WhenCommandExceedsFrameMax_FailsOnlyThatCommandAndStaysReady`.
+- `EventStreamServiceTests.StreamEventsAsync_WithConcurrentStreams_TracksAggregateQueueDepth`
+  — polls a metric against a five-second deadline. Its helper now reports the unmet condition
+  rather than letting a bare `TaskCanceledException` escape, so a load-induced timeout here
+  names what it was waiting for instead of looking like an unexplained cancellation.
+
+A failure in that list is evidence about machine load, not about the change under test. Check
+for a concurrent x86 build/test (`Get-Process dotnet, testhost, testhost.net48.x86,
+MSBuild, VBCSCompiler`) and re-run the affected class on its own before treating it as real.
+windev has 36 logical CPUs and `xunit.runner.json` sets `maxParallelThreads: -1`, so the
+suite runs far wider there than on the macOS dev box — that width is what turns these
+real-clock deadlines into failures.
+
+### The full-suite testhost does not exit on windev
+
+After the last test completes, the x64 `testhost` process stops doing work but never exits,
+so `dotnet test` never returns and the run has to be killed. This does **not** happen on
+filtered runs (`--filter …`), which exit normally, and does not happen on the macOS dev box —
+it is specific to a full-suite run on windev. Run the full suite with a hang guard so the
+wedged host is torn down and the pass/fail summary is still printed:
+
+```powershell
+dotnet test src\ZB.MOM.WW.MxGateway.Tests\ZB.MOM.WW.MxGateway.Tests.csproj `
+    --blame-hang --blame-hang-timeout 5m --blame-hang-dump-type none
+```
+
+The summary line ahead of the abort is the real result; the process exit code is nonzero
+because of the abort even when every test passed, so read the summary rather than the exit
+code. Prefer filtered runs on windev whenever the change under test allows it.
+
 ## Continuous Integration
 
 CI runs on Gitea Actions (`.gitea/workflows/ci.yml`; origin is Gitea at

src/ZB.MOM.WW.MxGateway.Tests/Gateway/Grpc/EventStreamServiceTests.cs

@@ -699,12 +699,26 @@ public sealed class EventStreamServiceTests
         };
     }
 
-    private static async Task WaitUntilAsync(Func<bool> predicate)
+    // The real-clock deadline here is load-sensitive on a wide host (windev runs this suite
+    // 36-way parallel). Surfacing the unmet condition instead of letting the bare
+    // TaskCanceledException escape is what makes such a failure diagnosable rather than a
+    // mystery cancellation attributed to "the environment".
+    private static async Task WaitUntilAsync(
+        Func<bool> predicate,
+        [CallerArgumentExpression(nameof(predicate))] string? predicateExpression = null)
     {
         using CancellationTokenSource cancellationTokenSource = new(TestTimeout);
         while (!predicate())
         {
-            await Task.Delay(TimeSpan.FromMilliseconds(10), cancellationTokenSource.Token);
+            try
+            {
+                await Task.Delay(TimeSpan.FromMilliseconds(10), cancellationTokenSource.Token);
+            }
+            catch (OperationCanceledException)
+            {
+                Assert.Fail(
+                    $"Timed out after {TestTimeout} waiting for condition: {predicateExpression}");
+            }
         }
     }
 

src/ZB.MOM.WW.MxGateway.Tests/Gateway/Sessions/SessionManagerTests.cs

@@ -61,10 +61,20 @@ public sealed class SessionManagerTests
 
         // 104-byte sun_path − NUL − ~49-char default macOS TMPDIR − "CoreFxPipe_".
         const int MaxPipeNameLength = 104 - 1 - 49 - 11;
-        int worstCasePidDigits = 5 - Environment.ProcessId.ToString(System.Globalization.CultureInfo.InvariantCulture).Length;
+
+        // macOS pids top out at 99999, so 5 digits is the worst case the budget must survive.
+        // The check must substitute that worst case for the *running* pid's digit count rather
+        // than pad the measured length upward: Windows pids are routinely 6 digits, which would
+        // otherwise fail this assertion on a host whose own pipe-name limit (256) is irrelevant
+        // to the macOS budget being guarded here.
+        const int WorstCaseMacOsPidDigits = 5;
+        int runningPidDigits = Environment.ProcessId
+            .ToString(System.Globalization.CultureInfo.InvariantCulture).Length;
+        int worstCaseLength = session.PipeName.Length - runningPidDigits + WorstCaseMacOsPidDigits;
         Assert.True(
-            session.PipeName.Length + Math.Max(0, worstCasePidDigits) <= MaxPipeNameLength,
-            $"Pipe name '{session.PipeName}' would overflow the macOS socket-path budget at a 5-digit pid.");
+            worstCaseLength <= MaxPipeNameLength,
+            $"Pipe name '{session.PipeName}' would overflow the macOS socket-path budget at a 5-digit pid " +
+            $"({worstCaseLength} > {MaxPipeNameLength}).");
     }
 
     /// <summary>Verifies that a session opened by an authenticated caller records that caller's API key id in OwnerKeyId.</summary>

src/ZB.MOM.WW.MxGateway.Tests/Security/Tls/SelfSignedCertificateProviderTests.cs

@@ -1,3 +1,4 @@
+using System.Net;
 using System.Security.Cryptography.X509Certificates;
 using Microsoft.Extensions.Logging.Abstractions;
 using Microsoft.Extensions.Time.Testing;
@@ -25,15 +26,19 @@ public sealed class SelfSignedCertificateProviderTests
         Assert.True(cert.NotBefore.ToUniversalTime() < time.GetUtcNow().UtcDateTime);
         Assert.True(cert.HasPrivateKey);
 
-        string sans = ReadSubjectAltNames(cert);
-        Assert.Contains("localhost", sans);
-        Assert.Contains("gw.internal", sans);
-        Assert.Contains(Environment.MachineName, sans);
-        // Format() renders IP SANs as "IP Address:<addr>"; the IPv6 loopback may appear
-        // as "::1" or its expanded form depending on the platform crypto library.
-        Assert.Contains("127.0.0.1", sans);
-        Assert.True(sans.Contains("::1") || sans.Contains("0:0:0:0:0:0:0:1"),
-            $"Expected IPv6 loopback in SANs but got: {sans}");
+        X509SubjectAlternativeNameExtension san = ReadSubjectAltNames(cert);
+        string[] dnsNames = [.. san.EnumerateDnsNames()];
+        IPAddress[] ipAddresses = [.. san.EnumerateIPAddresses()];
+
+        // DNS SANs are compared case-insensitively (DNS names are), and IP SANs are compared
+        // as parsed IPAddress values. Asserting against the extension's Format() string instead
+        // would be platform-dependent: Windows' CryptFormatObject renders the IPv6 loopback
+        // fully expanded ("0000:0000:...:0001") while the managed formatter renders "::1".
+        Assert.Contains(dnsNames, name => name.Equals("localhost", StringComparison.OrdinalIgnoreCase));
+        Assert.Contains(dnsNames, name => name.Equals("gw.internal", StringComparison.OrdinalIgnoreCase));
+        Assert.Contains(dnsNames, name => name.Equals(Environment.MachineName, StringComparison.OrdinalIgnoreCase));
+        Assert.Contains(IPAddress.Loopback, ipAddresses);
+        Assert.Contains(IPAddress.IPv6Loopback, ipAddresses);
 
         X509EnhancedKeyUsageExtension eku = cert.Extensions.OfType<X509EnhancedKeyUsageExtension>().Single();
         Assert.Contains(eku.EnhancedKeyUsages.Cast<System.Security.Cryptography.Oid>(),
@@ -155,8 +160,6 @@ public sealed class SelfSignedCertificateProviderTests
 
     private const string SubjectAltNameOid = "2.5.29.17";
 
-    private static string ReadSubjectAltNames(X509Certificate2 cert)
-        => cert.Extensions
-            .First(e => e.Oid?.Value == SubjectAltNameOid)
-            .Format(false);
+    private static X509SubjectAlternativeNameExtension ReadSubjectAltNames(X509Certificate2 cert)
+        => new(cert.Extensions.First(e => e.Oid?.Value == SubjectAltNameOid).RawData);
 }