Joseph Doherty
ae91e6b7f7
#816 finding: the SDK checks that the server certificate names the host only when a session is created with checkDomain:true. The driver, the AdminUI browser and Client.Shared all called the 8-argument DefaultSessionFactory.CreateAsync, which forwards checkDomain:false — so no host-name check ever ran (why a trusted NAT cert for *.invalid browsed fine over an IP in #813). When enabled the SDK (Session.OpenAsync -> ValidateDomains -> FindDomain) compares ConfiguredEndpoint.EndpointUrl.IdnHost — the CONFIGURED host, because OpcUaEndpointSelection/EndpointSelector rewrote the advertised one — against the cert's DNS/IP SANs; secured sessions only, and the auto-accept handler accepts a mismatch. Policy: certificateValidation.validateServerHostName, tri-state. Unset = OpcUaCertificateValidationOptions.DefaultValidateServerHostName = false, the pre-#816 behaviour, so nothing changes on upgrade; true = validate the configured host; false = explicit NAT/port-publish opt-out. Threaded into the driver, the AdminUI browser (same JSON) and the AdminUI form (a select that keeps "unset" unset), Client.Shared ConnectionSettings.ValidateServerHostName and the CLI --validate-host-name. A driver refusal names the opt-out. #818: the NAT fixture's /app/pki is on the opc-plc-nat-pki volume (verified: same thumbprint across --force-recreate), its PUBLIC cert is committed as Docker/nat/opc-plc-nat-server.der, and NatPublishedEndpointBrowseTests run every secure browse on a throwaway PKI (internal browser seam): refusal with nothing trusted, positive with the cert trusted (unset/false host check), and a BadCertificateHostNameInvalid refusal with the check on. The trusted cert is OPCUA_NAT_SERVER_CERT or the committed one (skip on thumbprint mismatch); OPCUA_NAT_SERVER_CERT_TRUSTED is retired. CI exports its throwaway NAT container's public cert. ServerHostNameValidationTests pin the driver against the plain opc-plc. MIN_EXECUTED not raised (needs a hosted run): +6 in v2-integration (440 -> 446), documented in scripts/ci/README.md.
94 lines
6.5 KiB
Markdown
94 lines
6.5 KiB
Markdown
# Whole-solution CI
|
|
|
|
The `v2-ci` workflow restores, builds and tests in one job so the test step uses
|
|
that job's Release binaries. Restore disables parallel package work; restore and
|
|
build use at most two MSBuild workers. Testing uses one MSBuild worker, disables
|
|
parallel target-framework runs and loads `ci.runsettings`: one VSTest process and
|
|
two xUnit collection workers with conservative scheduling. Assemblies that disable
|
|
parallelization keep that setting.
|
|
|
|
The workflow retains the E2E/LiveIntegration exclusions, includes SlowUnit, keeps
|
|
Akka's test time factor at three, and requires at least 12,000 executed tests.
|
|
Runner capacity and TRX files remain available for diagnosing failures. These
|
|
limits reduce contention; they do not prove every timing failure has one cause.
|
|
There are no automatic test retries or changed assertion deadlines.
|
|
|
|
Gitea 1.26 supports the repository-wide `lmxopcua-v2-ci` concurrency group. Only
|
|
one run is active; an active run is not cancelled, but a newer pending run may
|
|
supersede an older pending run. The group does not serialize other repositories
|
|
using the same host.
|
|
|
|
Stall guards (#718): the job has `timeout-minutes: 120` (healthy runs take about 60 minutes)
|
|
and the test step passes `--blame-hang --blame-hang-timeout 20m --blame-hang-dump-type none`, so a
|
|
test that stops making progress is killed and named in the log instead of leaving hours of silence
|
|
(run 959 stalled 3h12m until the server reaper failed it).
|
|
|
|
To reproduce the bounded test leg after restore and Release build:
|
|
|
|
```bash
|
|
AKKA_TEST_TIMEFACTOR=3 dotnet test ZB.MOM.WW.OtOpcUa.slnx \
|
|
--configuration Release --no-build -m:1 -p:TestTfmsInParallel=false \
|
|
--settings scripts/ci/ci.runsettings \
|
|
--filter "Category!=E2E&Category!=LiveIntegration" \
|
|
--logger trx --results-directory artifacts/trx
|
|
MIN_EXECUTED=12000 bash scripts/ci/assert-not-all-skipped.sh artifacts/trx/*.trx
|
|
```
|
|
|
|
The guard uses the existing TRX `Counters/@executed` value for the aggregate
|
|
floor. Its `skipped` diagnostic counts individual `Results/UnitTestResult`
|
|
records whose outcome is `NotExecuted`, using the document's XML namespace.
|
|
Some VSTest reports set summary `notExecuted` to zero even when these records
|
|
are present. Skip records are counted per file, including repeated runs; they
|
|
are not a count of unique test methods and do not affect the executed floor.
|
|
For malformed XML, the diagnostic retains the old summary-counter fallback;
|
|
this guard does not add XML validation to its existing gate policy.
|
|
|
|
Run the deterministic script fixtures without building or running the solution:
|
|
|
|
```bash
|
|
python3 scripts/ci/test-assert-not-all-skipped.py
|
|
```
|
|
|
|
Verified hosted baseline: [Gitea run 975](https://gitea.dohertylan.com/dohertj2/lmxopcua/actions/runs/975)
|
|
completed successfully on 2026-10-13 for commit
|
|
`76bdca90e94e814e06ef10b78271e2b492b2ba32` (merged by PR #711).
|
|
Restore, Release build, README drift check, the serialized whole-solution test
|
|
step, the 12,000-test floor and TRX upload all passed. The floor counted
|
|
14,630 executed tests across 62 TRX files. This baseline predates the individual
|
|
skip-reporting correction; its printed zero skip diagnostic is not skip evidence.
|
|
|
|
## Fixture-backed integration and guarded E2E workflows (#757)
|
|
|
|
**Both are UNVERIFIED on a hosted Gitea runner until their first run**; they were validated locally only
|
|
(YAML parse, the exact `dotnet test` commands and fixture script against local throwaway containers).
|
|
Neither touches `v2-ci.yml`; each has its own concurrency group (`lmxopcua-v2-integration`,
|
|
`lmxopcua-v2-e2e`, `cancel-in-progress: false`) so a pending run of one cannot supersede a pending
|
|
`lmxopcua-v2-ci` run.
|
|
|
|
- **`v2-integration.yml`** (push to `master`, path-filtered PRs, manual; `timeout-minutes: 60`).
|
|
`integration-fixtures.sh up` starts throwaway SQL Server 2022, PostgreSQL 17 (seeded from `seed-pg.sql`),
|
|
QuestDB and opc-plc — plus a port-NAT opc-plc (`otopcua-ci-opcplc-nat`, host 50110 → 50000, private `.invalid`
|
|
advertised host, `OPCUA_NAT_ENDPOINT`, #813) — (`otopcua-ci-*`, non-default host ports), exports the env vars the fixtures read, and
|
|
`down` removes them. It uses plain `docker run`, not `services:` (act_runner support is undocumented and
|
|
Postgres needs a seed step); the only runner requirement is a docker daemon. Runs the Sql, Historian.SqlServer,
|
|
Historian.QuestDb, OpcUaClient and OpcUaClient.Browser `*.IntegrationTests` with `MIN_EXECUTED=440`.
|
|
Local run: 468 executed / 8 skipped (Oracle x2, the blackhole `docker pause` gate, one QuestDB stress test,
|
|
4 others), about 4.5 min of test time, QuestDB the longest (~3.3 min). Not covered (no pullable image or
|
|
needs credentials): Modbus/S7/AB/FOCAS/TwinCAT sims, Mqtt, MTConnect, Oracle, Host real-infra mode.
|
|
**#816/#818 added 6 executing tests here, floor NOT yet raised:** `ServerHostNameValidationTests` (3, vs the plain
|
|
`otopcua-ci-opcplc`) and the NAT positive-secure / host-name cases (3, which run because `integration-fixtures.sh`
|
|
now exports the NAT container's public certificate as `OPCUA_NAT_SERVER_CERT`). Locally (2026-11-11, shared host)
|
|
the two OpcUaClient projects went 13 → 19 executed with 0 skipped, and `assert-not-all-skipped.sh` counted them.
|
|
Once a hosted run's TRX shows `Driver.OpcUaClient.trx` executed=8 and `Driver.OpcUaClient.Browser.trx` executed=11,
|
|
raise `MIN_EXECUTED` in `v2-integration.yml` **440 → 446**. If the NAT three show as skipped there, the certificate
|
|
hand-off (`docker exec … /app/pki/own/certs/*.der`) did not work on the runner — fix that, do not lower the bump.
|
|
- **`v2-e2e.yml`** (manual + weekly cron only; `timeout-minutes: 90`). `probe-e2e-fixtures.sh` TCP-probes
|
|
10.100.0.35 ports 5020/1102/50000; if any is closed every later step is skipped and the run is green with a
|
|
notice. Otherwise runs the Modbus, S7, OpcUaClient and OpcUaClient.Browser integration projects against the
|
|
shared fleet with `MIN_EXECUTED=35` (`OPCUA_NAT_ENDPOINT` points at the shared NAT stack on :50010; its 6 tests
|
|
skip if it is down, so it is not a probe-required port — and its 3 certificate-trusting ones also skip if the stack's
|
|
certificate no longer matches the committed `Docker/nat/opc-plc-nat-server.der`, so do not count them in the floor;
|
|
#816's 3 `ServerHostNameValidationTests` need only the probe-required :50000, so this floor may go **35 → 38** after
|
|
a hosted run shows them executed) (local: 42 executed / 38 skipped; Modbus skips the profiles that are not
|
|
up). No test carries `Category=E2E` today, so the zero-match guard is this executed floor.
|