Joseph Doherty
f13f35bc79
IntegrationTests-003: the live MXAccess smoke test asserted on the first streamed event, which a registration/quality bootstrap event could occupy. The recording writer now waits for the first event matching a predicate (Family == OnDataChange). IntegrationTests-004: the cleanup `finally` could throw and mask an original assertion failure. Shutdown now routes through a helper that logs cleanup exceptions instead of propagating them. IntegrationTests-005: added live MXAccess parity tests — a Write round-trip to an advised item, and an invalid-handle command surfacing the MXAccess failure without a transport fault. IntegrationTests-006: added live LDAP failure-path tests — wrong password (no password leak), unknown username, and server-unreachable. docs/GatewayTesting.md updated to describe the new cases. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
238 lines
11 KiB
Markdown
238 lines
11 KiB
Markdown
# Gateway Testing
|
|
|
|
Gateway tests run without installed MXAccess by using fake workers, fake
|
|
transports, and in-process gRPC service fakes. Live MXAccess verification belongs
|
|
in opt-in integration tests because it depends on installed COM components and
|
|
provider state.
|
|
|
|
## Fake Worker Harness
|
|
|
|
`FakeWorkerHarness` in `src/MxGateway.Tests/Gateway/Workers/Fakes/` provides an
|
|
in-process worker side for named-pipe IPC tests. It uses the same
|
|
`WorkerFrameReader`, `WorkerFrameWriter`, and `WorkerEnvelope` contract as the
|
|
gateway so tests exercise real frame validation and worker-client state changes.
|
|
|
|
Use the harness when a gateway or session test needs worker behavior without
|
|
starting `MxGateway.Worker.exe` or loading MXAccess COM. The harness scripts:
|
|
|
|
- `WorkerHello` and `WorkerReady` startup,
|
|
- command replies with matching correlation ids,
|
|
- ordered `WorkerEvent` frames,
|
|
- `WorkerHeartbeat` frames,
|
|
- `WorkerFault` frames,
|
|
- shutdown acknowledgements,
|
|
- malformed protobuf payloads and oversized frame headers,
|
|
- slow or hung workers by withholding a reply.
|
|
|
|
Session-level tests can connect the harness to the pipe created by
|
|
`SessionWorkerClientFactory` with `ConnectToGatewayPipeAsync`. Lower-level
|
|
`WorkerClient` tests can use `CreateConnectedPairAsync` to create both pipe ends
|
|
inside the test.
|
|
|
|
`GatewayEndToEndFakeWorkerSmokeTests` composes the real gRPC service,
|
|
`SessionManager`, `SessionWorkerClientFactory`, `WorkerClient`, and
|
|
`EventStreamService` with a scripted fake worker launcher. The smoke test covers
|
|
`OpenSession`, `Register`, `AddItem`, `Advise`, one streamed `OnDataChange`
|
|
event, and `CloseSession` without loading MXAccess COM.
|
|
|
|
## Live MXAccess Smoke
|
|
|
|
`WorkerLiveMxAccessSmokeTests` in `src/MxGateway.IntegrationTests/` composes the
|
|
real gRPC service, `SessionManager`, `SessionWorkerClientFactory`,
|
|
`WorkerClient`, `WorkerProcessLauncher`, and `MxGateway.Worker.exe`. It is
|
|
skipped unless `MXGATEWAY_RUN_LIVE_MXACCESS_TESTS=1` is set because it creates
|
|
the installed MXAccess COM object and depends on live provider state.
|
|
|
|
The live smoke opens a gateway session, launches the x86 worker, runs
|
|
`Register`, `AddItem`, and `Advise`, waits a bounded time for the first
|
|
`OnDataChange` event (skipping any earlier bootstrap/registration-state event),
|
|
and closes the session in a `finally` block so the worker gets a graceful
|
|
shutdown request even when a command or event assertion fails. Cleanup failures
|
|
in that `finally` block are logged rather than thrown, so a real assertion
|
|
failure is never masked by a shutdown timeout.
|
|
|
|
`WorkerLiveMxAccessSmokeTests` additionally covers two MXAccess parity paths the
|
|
fake-worker tests cannot validate:
|
|
|
|
- a `Write` round-trip against an advised item, and
|
|
- an `AddItem` against an invalid server handle, asserting the MXAccess failure
|
|
surfaces in the command reply without faulting the gateway transport.
|
|
|
|
All three tests are gated by the same `MXGATEWAY_RUN_LIVE_MXACCESS_TESTS=1`
|
|
opt-in variable.
|
|
|
|
Build the worker before running the smoke:
|
|
|
|
```bash
|
|
dotnet build src/MxGateway.Worker/MxGateway.Worker.csproj -p:Platform=x86
|
|
```
|
|
|
|
Run the smoke explicitly:
|
|
|
|
```bash
|
|
$env:MXGATEWAY_RUN_LIVE_MXACCESS_TESTS = "1"
|
|
dotnet test src/MxGateway.IntegrationTests/MxGateway.IntegrationTests.csproj --filter FullyQualifiedName~WorkerLiveMxAccessSmokeTests
|
|
```
|
|
|
|
Optional live smoke variables:
|
|
|
|
| Variable | Default | Description |
|
|
|----------|---------|-------------|
|
|
| `MXGATEWAY_LIVE_MXACCESS_WORKER_EXE` | First existing `MxGateway.Worker.exe` under `src/MxGateway.Worker/bin/...` | Worker executable path. Set this when running against a packaged worker or a non-default build output. |
|
|
| `MXGATEWAY_LIVE_MXACCESS_ITEM` | `TestChildObject.TestInt` | MXAccess item reference used by `AddItem`. |
|
|
| `MXGATEWAY_LIVE_MXACCESS_CLIENT_NAME` | `MxGateway.IntegrationTests` | Client name passed to `Register`. |
|
|
| `MXGATEWAY_LIVE_MXACCESS_EVENT_TIMEOUT_SECONDS` | `15` | Maximum wait for the first `OnDataChange`. |
|
|
|
|
The test output includes session id, worker process id, command status,
|
|
HRESULT/status diagnostics, event sequence and handles, close status, and worker
|
|
stdout/stderr lines emitted during the run.
|
|
|
|
## Live Galaxy Repository
|
|
|
|
`GalaxyRepositoryLiveTests` in `src/MxGateway.IntegrationTests/Galaxy/` exercises
|
|
`GalaxyRepository` directly against the `ZB` Galaxy Repository SQL database. It is
|
|
skipped unless `MXGATEWAY_RUN_LIVE_GALAXY_TESTS=1` is set because it depends on a
|
|
reachable SQL Server instance and deployed Galaxy state — fake-worker tests cannot
|
|
cover the SQL browse RPCs.
|
|
|
|
The suite covers `TestConnectionAsync`, `GetLastDeployTimeAsync`,
|
|
`GetHierarchyAsync`, and `GetAttributesAsync`. `GetHierarchyAsync` and
|
|
`GetAttributesAsync` assert a non-empty result, so the connected `ZB` database
|
|
must contain a deployed Galaxy, not just an empty schema.
|
|
|
|
Run the Galaxy live tests explicitly:
|
|
|
|
```bash
|
|
$env:MXGATEWAY_RUN_LIVE_GALAXY_TESTS = "1"
|
|
dotnet test src/MxGateway.IntegrationTests/MxGateway.IntegrationTests.csproj --filter FullyQualifiedName~GalaxyRepositoryLiveTests
|
|
```
|
|
|
|
Optional live Galaxy variables:
|
|
|
|
| Variable | Default | Description |
|
|
|----------|---------|-------------|
|
|
| `MXGATEWAY_LIVE_GALAXY_CONN` | `Server=localhost;Database=ZB;Integrated Security=True;TrustServerCertificate=True;Encrypt=False;` | Galaxy Repository connection string. Set this when the `ZB` database is on a non-default instance or needs SQL authentication. |
|
|
|
|
The default connection string targets `ZB` on `localhost` with Windows
|
|
authentication, which matches the Galaxy Repository conventions in CLAUDE.md.
|
|
|
|
## Live LDAP
|
|
|
|
`DashboardLdapLiveTests` in `src/MxGateway.IntegrationTests/` exercises
|
|
`DashboardAuthenticator` against the live GLAuth directory. It is skipped unless
|
|
`MXGATEWAY_RUN_LIVE_LDAP_TESTS=1` is set because it binds against the GLAuth
|
|
service described in `glauth.md`.
|
|
|
|
The suite builds the authenticator with a default `GatewayOptions`, so
|
|
`LdapOptions.RequiredGroup` keeps its `GwAdmin` default. `GwAdmin` is the
|
|
gateway-specific dashboard-admin role and is **not** part of the five baseline
|
|
GLAuth role groups — it must be provisioned before the LDAP live tests pass.
|
|
`AuthenticateAsync_AdminInGwAdminGroup_Succeeds` fails (rather than skips) when
|
|
GLAuth has only the baseline groups, so this is a hard prerequisite beyond "LDAP
|
|
is up." See the "Adding a gw-specific group" section of `glauth.md` for the
|
|
provisioning step that adds `GwAdmin` and grants it to `admin`.
|
|
|
|
The suite covers both the success path and the `DashboardAuthenticator` failure
|
|
branches: `admin` in `GwAdmin` succeeds; `readonly` is denied for missing group;
|
|
`admin` with a wrong password is rejected by the candidate bind without leaking
|
|
the password into `FailureMessage`; an unknown username yields no candidate; and
|
|
an unreachable LDAP server is absorbed into a failed result rather than throwing.
|
|
|
|
Run the LDAP live tests explicitly:
|
|
|
|
```bash
|
|
$env:MXGATEWAY_RUN_LIVE_LDAP_TESTS = "1"
|
|
dotnet test src/MxGateway.IntegrationTests/MxGateway.IntegrationTests.csproj --filter FullyQualifiedName~DashboardLdapLiveTests
|
|
```
|
|
|
|
## Client E2E Scripts
|
|
|
|
`scripts/discover-testmachine-tags.ps1` queries the ZB Galaxy Repository for the
|
|
deployed runtime references used by the live client e2e scripts. It reads
|
|
`TestMachine_001` through `TestMachine_020` and the expected attributes:
|
|
|
|
- `ProtectedValue`
|
|
- `TestChangingInt`
|
|
- `TestBoolArray`
|
|
- `TestIntArray`
|
|
- `TestDateTimeArray`
|
|
- `TestStringArray`
|
|
|
|
The discovery output includes the exact `fullTagReference`, data type, array
|
|
dimension, and security classification. The array attributes are expected to be
|
|
dimension 50. `ProtectedValue` has security classification 2 and requires
|
|
secured write semantics; the current client CLI e2e runner subscribes to it but
|
|
does not attempt a normal `Write`.
|
|
|
|
Run discovery directly when validating the Galaxy Repository inputs:
|
|
|
|
```powershell
|
|
powershell -ExecutionPolicy Bypass -File scripts/discover-testmachine-tags.ps1 -Json
|
|
```
|
|
|
|
`scripts/run-client-e2e-tests.ps1` drives the .NET, Go, Rust, Python, and Java
|
|
client CLIs through a live gateway session. For each client it opens one
|
|
session, registers, verifies `SubscribeBulk` and `UnsubscribeBulk` on a bounded
|
|
tag subset, adds and advises every discovered test tag, reads a bounded event
|
|
stream, then closes the session in a `finally` path. The script writes a JSON
|
|
report under `artifacts/e2e/`.
|
|
|
|
Build the gateway and worker, start the gateway, and provide a valid API key
|
|
before running the client e2e script:
|
|
|
|
```powershell
|
|
$env:MXGATEWAY_API_KEY = "<api-key>"
|
|
powershell -ExecutionPolicy Bypass -File scripts/run-client-e2e-tests.ps1
|
|
```
|
|
|
|
Useful runner options:
|
|
|
|
```powershell
|
|
powershell -ExecutionPolicy Bypass -File scripts/run-client-e2e-tests.ps1 -Clients dotnet,python -MachineStart 1 -MachineEnd 2
|
|
powershell -ExecutionPolicy Bypass -File scripts/run-client-e2e-tests.ps1 -BulkTagCount 10
|
|
powershell -ExecutionPolicy Bypass -File scripts/run-client-e2e-tests.ps1 -SkipStream
|
|
powershell -ExecutionPolicy Bypass -File scripts/run-client-e2e-tests.ps1 -SkipBulk
|
|
powershell -ExecutionPolicy Bypass -File scripts/run-client-e2e-tests.ps1 -Endpoint localhost:5000 -ApiKeyEnv MXGATEWAY_API_KEY
|
|
```
|
|
|
|
## Focused Commands
|
|
|
|
Run the cross-language smoke matrix tests after changing the documented client
|
|
smoke command list:
|
|
|
|
```bash
|
|
dotnet test src/MxGateway.Tests/MxGateway.Tests.csproj --filter FullyQualifiedName~CrossLanguageSmokeMatrixTests
|
|
```
|
|
|
|
Run the parity fixture matrix tests after changing the integration parity
|
|
scenario list:
|
|
|
|
```bash
|
|
dotnet test src/MxGateway.Tests/MxGateway.Tests.csproj --filter FullyQualifiedName~ParityFixtureMatrixTests
|
|
```
|
|
|
|
Run the fake worker tests after changing gateway worker IPC, session startup, or
|
|
event streaming behavior:
|
|
|
|
```bash
|
|
dotnet test src/MxGateway.Tests/MxGateway.Tests.csproj --filter FullyQualifiedName~FakeWorkerHarnessTests
|
|
dotnet test src/MxGateway.Tests/MxGateway.Tests.csproj --filter FullyQualifiedName~SessionWorkerClientFactoryFakeWorkerTests
|
|
dotnet test src/MxGateway.Tests/MxGateway.Tests.csproj --filter FullyQualifiedName~GatewayEndToEndFakeWorkerSmokeTests
|
|
dotnet test src/MxGateway.Tests/MxGateway.Tests.csproj --filter FullyQualifiedName~WorkerClientTests
|
|
dotnet test src/MxGateway.Worker.Tests/MxGateway.Worker.Tests.csproj -p:Platform=x86 --filter FullyQualifiedName~WorkerPipeSessionTests
|
|
```
|
|
|
|
Run the gateway test project after shared gateway test infrastructure changes:
|
|
|
|
```bash
|
|
dotnet test src/MxGateway.Tests/MxGateway.Tests.csproj
|
|
```
|
|
|
|
## Related Documentation
|
|
|
|
- [Cross-Language Smoke Matrix](./CrossLanguageSmokeMatrix.md)
|
|
- [Parity Fixture Matrix](./ParityFixtureMatrix.md)
|
|
- [Gateway Process Design](./GatewayProcessDesign.md)
|
|
- [Worker Frame Protocol](./WorkerFrameProtocol.md)
|
|
- [MXAccess Worker Instance Detailed Design](./MxAccessWorkerInstanceDesign.md)
|