test(windev): fix the two Windows-only gateway test failures dismissed as environmental
Both failures in the windev baseline were test bugs that reproduce on any Windows host, not anything missing or misconfigured on windev. SelfSignedCertificateProviderTests.GenerateCertificate_HasExpectedSansEkuAndValidity asserted SAN content by substring-matching X509Extension.Format(false). That string comes from the platform crypto library: Windows' CryptFormatObject renders the IPv6 loopback fully expanded (0000:0000:...:0001) where the managed formatter renders "::1", so the loopback assertion could never hold on Windows. Decode the extension with X509SubjectAlternativeNameExtension and compare parsed IPAddress values and DNS names instead, which removes the platform-dependent formatting from the assertion. SessionManagerTests.OpenSessionAsync_PipeNameIsShortAndUniquePerPidAndSession guards the 104-byte macOS sun_path budget NEXT-01 shortened the pipe name to fit. It padded the measured name up to a five-digit pid but never substituted that worst case downward, so Windows' routine six-digit pids over-counted by a character against a budget that does not constrain the host running the test. Substitute the five-digit macOS worst case for the running pid's digit count so the check measures the name format rather than the current pid. EventStreamServiceTests.WaitUntilAsync now reports the unmet condition on timeout instead of letting a bare TaskCanceledException escape. Its five-second real-clock deadline is genuinely load-sensitive on windev (36 logical CPUs, maxParallelThreads -1), and an opaque cancellation there is exactly what got the previous failures filed as "environmental" and left unexplained. Documents the windev run in docs/GatewayTesting.md: the two fixed bugs and their root causes, the real-pipe suites whose failures are evidence of machine load rather than of the change under test, and the full-suite testhost that completes every test and then never exits (filtered runs exit normally; macOS exits cleanly). Corrects the CLAUDE.md claim that the suite exits cleanly on the Windows dev box.
This commit is contained in:
@@ -124,7 +124,7 @@ powershell -ExecutionPolicy Bypass -File scripts/run-client-e2e-tests.ps1
|
||||
|
||||
When source code changes, build and test the affected component before reporting work done. If the change crosses component boundaries, build each affected component — don't rely on a single top-level build:
|
||||
|
||||
**Run targeted tests per task, never the full suite each time.** When executing a plan task-by-task, run only the tests that exercise the code that task touched (`dotnet test --filter "FullyQualifiedName~<TestClass>"`, or the per-task test named in the plan). The full gateway suite is slow — run it at most once per phase (after a related batch of tasks lands), not after every task. This is a speed guideline, not a correctness one: the suite exits cleanly (verified on macOS and the Windows dev box — 0 surviving `testhost`/worker processes after a full run), so filtered runs are about turnaround, not about avoiding a process leak.
|
||||
**Run targeted tests per task, never the full suite each time.** When executing a plan task-by-task, run only the tests that exercise the code that task touched (`dotnet test --filter "FullyQualifiedName~<TestClass>"`, or the per-task test named in the plan). The full gateway suite is slow — run it at most once per phase (after a related batch of tasks lands), not after every task. This is a speed guideline on macOS, where the suite exits cleanly (0 surviving `testhost`/worker processes after a full run) — filtered runs there are about turnaround, not about avoiding a process leak. **On windev a full-suite run wedges**: every test completes, then the x64 `testhost` never exits and `dotnet test` never returns (filtered runs exit normally). Run the full suite there with `--blame-hang --blame-hang-timeout 5m --blame-hang-dump-type none` and read the summary line, not the exit code — see "Running the Gateway Suite on windev" in `docs/GatewayTesting.md`.
|
||||
|
||||
| Changed area | Required verification |
|
||||
|---|---|
|
||||
|
||||
@@ -485,6 +485,88 @@ Run the gateway test project after shared gateway test infrastructure changes:
|
||||
dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj
|
||||
```
|
||||
|
||||
## Running the Gateway Suite on windev
|
||||
|
||||
The gateway suite (`ZB.MOM.WW.MxGateway.Tests`, net10.0/x64) is not part of the CI
|
||||
Windows tier — `windows-x86` and `nightly-windev` run only the x86 Worker build and
|
||||
`Worker.Tests`. It is still run on windev by hand when a change needs Windows
|
||||
confirmation, and that run has three Windows-specific characteristics worth knowing
|
||||
before results are interpreted.
|
||||
|
||||
Run it from an isolated clone under `C:\build` checked out to the SHA under test — never
|
||||
the dirty Desktop checkout, and never the CI clone `C:\build\mxaccessgw-ci`, whose worktree
|
||||
lock belongs to the Worker tier.
|
||||
|
||||
Baseline on an otherwise idle windev (2026-08-10): **855 passed, 0 failed, 29 s**. The
|
||||
suite is smaller there than the 879 the macOS box runs because some cases are gated to
|
||||
Unix. Any failure is therefore a real signal — but read the load caveat below before
|
||||
acting on one.
|
||||
|
||||
### Two long-standing "windev-environmental" failures were test bugs, not the environment
|
||||
|
||||
Both were dismissed as environmental for months and are now fixed. Neither depended on
|
||||
anything installed on windev; both failed on **any** Windows host:
|
||||
|
||||
- `SelfSignedCertificateProviderTests.GenerateCertificate_HasExpectedSansEkuAndValidity`
|
||||
asserted SAN content by substring-matching `X509Extension.Format(false)`. That string is
|
||||
produced by the platform crypto library: Windows' `CryptFormatObject` renders the IPv6
|
||||
loopback fully expanded (`IP Address=0000:0000:0000:0000:0000:0000:0000:0001`) while the
|
||||
managed formatter used on macOS/Linux renders `::1`, so the loopback assertion failed on
|
||||
Windows only. The test now decodes the extension with `X509SubjectAlternativeNameExtension`
|
||||
and compares parsed `IPAddress` values and DNS names (case-insensitively, as DNS names
|
||||
are), which is platform-independent.
|
||||
- `SessionManagerTests.OpenSessionAsync_PipeNameIsShortAndUniquePerPidAndSession` guards the
|
||||
104-byte macOS `sun_path` budget that NEXT-01 shortened the pipe name to fit. It padded the
|
||||
measured name up to a five-digit pid but never substituted that worst case *downward*, so a
|
||||
six-digit pid — routine on Windows, impossible on macOS, where pids stop at 99999 — made the
|
||||
name one character "too long" against a budget that does not apply to the host running the
|
||||
test. The check now replaces the running pid's digit count with the five-digit macOS worst
|
||||
case, so it measures the name *format* rather than the current process's pid.
|
||||
|
||||
### The real-pipe suites are load-sensitive
|
||||
|
||||
These suites drive real named pipes against a five-second worker startup timeout and start
|
||||
failing when windev is busy — most often when the x86 Worker tier is building or testing at
|
||||
the same time. All five passed in the idle baseline above and all five failed in a run taken
|
||||
while an x86 build and `Worker.Tests` were in flight (that run also took 2 m 21 s against the
|
||||
idle 29 s):
|
||||
|
||||
- `GatewayEndToEndFakeWorkerSmokeTests`, `GatewayEndToEndMultiSubscriberTests`,
|
||||
`GatewayEndToEndReconnectReplayTests` — fail as
|
||||
`RpcException Status(StatusCode="Unavailable", Detail="Failed to open session …")`.
|
||||
- `SessionWorkerClientFactoryFakeWorkerTests.CreateAsync_WhenFakeWorkerStartupFails_ThrowsWorkerClientException`
|
||||
— the startup timeout beats the protocol violation the test is asserting, so the observed
|
||||
exception is `TimeoutException` instead of `WorkerClientException`.
|
||||
- `WorkerClientTests.InvokeAsync_WhenCommandExceedsFrameMax_FailsOnlyThatCommandAndStaysReady`.
|
||||
- `EventStreamServiceTests.StreamEventsAsync_WithConcurrentStreams_TracksAggregateQueueDepth`
|
||||
— polls a metric against a five-second deadline. Its helper now reports the unmet condition
|
||||
rather than letting a bare `TaskCanceledException` escape, so a load-induced timeout here
|
||||
names what it was waiting for instead of looking like an unexplained cancellation.
|
||||
|
||||
A failure in that list is evidence about machine load, not about the change under test. Check
|
||||
for a concurrent x86 build/test (`Get-Process dotnet, testhost, testhost.net48.x86,
|
||||
MSBuild, VBCSCompiler`) and re-run the affected class on its own before treating it as real.
|
||||
windev has 36 logical CPUs and `xunit.runner.json` sets `maxParallelThreads: -1`, so the
|
||||
suite runs far wider there than on the macOS dev box — that width is what turns these
|
||||
real-clock deadlines into failures.
|
||||
|
||||
### The full-suite testhost does not exit on windev
|
||||
|
||||
After the last test completes, the x64 `testhost` process stops doing work but never exits,
|
||||
so `dotnet test` never returns and the run has to be killed. This does **not** happen on
|
||||
filtered runs (`--filter …`), which exit normally, and does not happen on the macOS dev box —
|
||||
it is specific to a full-suite run on windev. Run the full suite with a hang guard so the
|
||||
wedged host is torn down and the pass/fail summary is still printed:
|
||||
|
||||
```powershell
|
||||
dotnet test src\ZB.MOM.WW.MxGateway.Tests\ZB.MOM.WW.MxGateway.Tests.csproj `
|
||||
--blame-hang --blame-hang-timeout 5m --blame-hang-dump-type none
|
||||
```
|
||||
|
||||
The summary line ahead of the abort is the real result; the process exit code is nonzero
|
||||
because of the abort even when every test passed, so read the summary rather than the exit
|
||||
code. Prefer filtered runs on windev whenever the change under test allows it.
|
||||
|
||||
## Continuous Integration
|
||||
|
||||
CI runs on Gitea Actions (`.gitea/workflows/ci.yml`; origin is Gitea at
|
||||
|
||||
@@ -699,13 +699,27 @@ public sealed class EventStreamServiceTests
|
||||
};
|
||||
}
|
||||
|
||||
private static async Task WaitUntilAsync(Func<bool> predicate)
|
||||
// The real-clock deadline here is load-sensitive on a wide host (windev runs this suite
|
||||
// 36-way parallel). Surfacing the unmet condition instead of letting the bare
|
||||
// TaskCanceledException escape is what makes such a failure diagnosable rather than a
|
||||
// mystery cancellation attributed to "the environment".
|
||||
private static async Task WaitUntilAsync(
|
||||
Func<bool> predicate,
|
||||
[CallerArgumentExpression(nameof(predicate))] string? predicateExpression = null)
|
||||
{
|
||||
using CancellationTokenSource cancellationTokenSource = new(TestTimeout);
|
||||
while (!predicate())
|
||||
{
|
||||
try
|
||||
{
|
||||
await Task.Delay(TimeSpan.FromMilliseconds(10), cancellationTokenSource.Token);
|
||||
}
|
||||
catch (OperationCanceledException)
|
||||
{
|
||||
Assert.Fail(
|
||||
$"Timed out after {TestTimeout} waiting for condition: {predicateExpression}");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>Fake session manager for testing event streams.</summary>
|
||||
|
||||
@@ -61,10 +61,20 @@ public sealed class SessionManagerTests
|
||||
|
||||
// 104-byte sun_path − NUL − ~49-char default macOS TMPDIR − "CoreFxPipe_".
|
||||
const int MaxPipeNameLength = 104 - 1 - 49 - 11;
|
||||
int worstCasePidDigits = 5 - Environment.ProcessId.ToString(System.Globalization.CultureInfo.InvariantCulture).Length;
|
||||
|
||||
// macOS pids top out at 99999, so 5 digits is the worst case the budget must survive.
|
||||
// The check must substitute that worst case for the *running* pid's digit count rather
|
||||
// than pad the measured length upward: Windows pids are routinely 6 digits, which would
|
||||
// otherwise fail this assertion on a host whose own pipe-name limit (256) is irrelevant
|
||||
// to the macOS budget being guarded here.
|
||||
const int WorstCaseMacOsPidDigits = 5;
|
||||
int runningPidDigits = Environment.ProcessId
|
||||
.ToString(System.Globalization.CultureInfo.InvariantCulture).Length;
|
||||
int worstCaseLength = session.PipeName.Length - runningPidDigits + WorstCaseMacOsPidDigits;
|
||||
Assert.True(
|
||||
session.PipeName.Length + Math.Max(0, worstCasePidDigits) <= MaxPipeNameLength,
|
||||
$"Pipe name '{session.PipeName}' would overflow the macOS socket-path budget at a 5-digit pid.");
|
||||
worstCaseLength <= MaxPipeNameLength,
|
||||
$"Pipe name '{session.PipeName}' would overflow the macOS socket-path budget at a 5-digit pid " +
|
||||
$"({worstCaseLength} > {MaxPipeNameLength}).");
|
||||
}
|
||||
|
||||
/// <summary>Verifies that a session opened by an authenticated caller records that caller's API key id in OwnerKeyId.</summary>
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
using System.Net;
|
||||
using System.Security.Cryptography.X509Certificates;
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using Microsoft.Extensions.Time.Testing;
|
||||
@@ -25,15 +26,19 @@ public sealed class SelfSignedCertificateProviderTests
|
||||
Assert.True(cert.NotBefore.ToUniversalTime() < time.GetUtcNow().UtcDateTime);
|
||||
Assert.True(cert.HasPrivateKey);
|
||||
|
||||
string sans = ReadSubjectAltNames(cert);
|
||||
Assert.Contains("localhost", sans);
|
||||
Assert.Contains("gw.internal", sans);
|
||||
Assert.Contains(Environment.MachineName, sans);
|
||||
// Format() renders IP SANs as "IP Address:<addr>"; the IPv6 loopback may appear
|
||||
// as "::1" or its expanded form depending on the platform crypto library.
|
||||
Assert.Contains("127.0.0.1", sans);
|
||||
Assert.True(sans.Contains("::1") || sans.Contains("0:0:0:0:0:0:0:1"),
|
||||
$"Expected IPv6 loopback in SANs but got: {sans}");
|
||||
X509SubjectAlternativeNameExtension san = ReadSubjectAltNames(cert);
|
||||
string[] dnsNames = [.. san.EnumerateDnsNames()];
|
||||
IPAddress[] ipAddresses = [.. san.EnumerateIPAddresses()];
|
||||
|
||||
// DNS SANs are compared case-insensitively (DNS names are), and IP SANs are compared
|
||||
// as parsed IPAddress values. Asserting against the extension's Format() string instead
|
||||
// would be platform-dependent: Windows' CryptFormatObject renders the IPv6 loopback
|
||||
// fully expanded ("0000:0000:...:0001") while the managed formatter renders "::1".
|
||||
Assert.Contains(dnsNames, name => name.Equals("localhost", StringComparison.OrdinalIgnoreCase));
|
||||
Assert.Contains(dnsNames, name => name.Equals("gw.internal", StringComparison.OrdinalIgnoreCase));
|
||||
Assert.Contains(dnsNames, name => name.Equals(Environment.MachineName, StringComparison.OrdinalIgnoreCase));
|
||||
Assert.Contains(IPAddress.Loopback, ipAddresses);
|
||||
Assert.Contains(IPAddress.IPv6Loopback, ipAddresses);
|
||||
|
||||
X509EnhancedKeyUsageExtension eku = cert.Extensions.OfType<X509EnhancedKeyUsageExtension>().Single();
|
||||
Assert.Contains(eku.EnhancedKeyUsages.Cast<System.Security.Cryptography.Oid>(),
|
||||
@@ -155,8 +160,6 @@ public sealed class SelfSignedCertificateProviderTests
|
||||
|
||||
private const string SubjectAltNameOid = "2.5.29.17";
|
||||
|
||||
private static string ReadSubjectAltNames(X509Certificate2 cert)
|
||||
=> cert.Extensions
|
||||
.First(e => e.Oid?.Value == SubjectAltNameOid)
|
||||
.Format(false);
|
||||
private static X509SubjectAlternativeNameExtension ReadSubjectAltNames(X509Certificate2 cert)
|
||||
=> new(cert.Extensions.First(e => e.Oid?.Value == SubjectAltNameOid).RawData);
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user