lmxopcua/scripts/ci/README.md

Whole-solution CI

The v2-ci workflow restores, builds and tests in one job so the test step uses that job's Release binaries. Restore disables parallel package work; restore and build use at most two MSBuild workers. Testing uses one MSBuild worker, disables parallel target-framework runs and loads ci.runsettings: one VSTest process and two xUnit collection workers with conservative scheduling. Assemblies that disable parallelization keep that setting.

The workflow retains the E2E/LiveIntegration exclusions, includes SlowUnit, keeps Akka's test time factor at three, and requires at least 12,000 executed tests. Runner capacity and TRX files remain available for diagnosing failures. These limits reduce contention; they do not prove every timing failure has one cause. There are no automatic test retries or changed assertion deadlines.

Gitea 1.26 supports the repository-wide lmxopcua-v2-ci concurrency group. Only one run is active; an active run is not cancelled, but a newer pending run may supersede an older pending run. The group does not serialize other repositories using the same host.

Stall guards (#718): the job has timeout-minutes: 120 (healthy runs take about 60 minutes) and the test step passes --blame-hang --blame-hang-timeout 20m --blame-hang-dump-type none, so a test that stops making progress is killed and named in the log instead of leaving hours of silence (run 959 stalled 3h12m until the server reaper failed it).

To reproduce the bounded test leg after restore and Release build:

AKKA_TEST_TIMEFACTOR=3 dotnet test ZB.MOM.WW.OtOpcUa.slnx \
  --configuration Release --no-build -m:1 -p:TestTfmsInParallel=false \
  --settings scripts/ci/ci.runsettings \
  --filter "Category!=E2E&Category!=LiveIntegration" \
  --logger trx --results-directory artifacts/trx
MIN_EXECUTED=12000 bash scripts/ci/assert-not-all-skipped.sh artifacts/trx/*.trx

The guard uses the existing TRX Counters/@executed value for the aggregate floor. Its skipped diagnostic counts individual Results/UnitTestResult records whose outcome is NotExecuted, using the document's XML namespace. Some VSTest reports set summary notExecuted to zero even when these records are present. Skip records are counted per file, including repeated runs; they are not a count of unique test methods and do not affect the executed floor. For malformed XML, the diagnostic retains the old summary-counter fallback; this guard does not add XML validation to its existing gate policy.

Run the deterministic script fixtures without building or running the solution:

python3 scripts/ci/test-assert-not-all-skipped.py

Verified hosted baseline: Gitea run 975 completed successfully on 2026-10-13 for commit 76bdca90e94e814e06ef10b78271e2b492b2ba32 (merged by PR #711). Restore, Release build, README drift check, the serialized whole-solution test step, the 12,000-test floor and TRX upload all passed. The floor counted 14,630 executed tests across 62 TRX files. This baseline predates the individual skip-reporting correction; its printed zero skip diagnostic is not skip evidence.

Fixture-backed integration and guarded E2E workflows (#757)

Both are UNVERIFIED on a hosted Gitea runner until their first run; they were validated locally only (YAML parse, the exact dotnet test commands and fixture script against local throwaway containers). Neither touches v2-ci.yml; each has its own concurrency group (lmxopcua-v2-integration, lmxopcua-v2-e2e, cancel-in-progress: false) so a pending run of one cannot supersede a pending lmxopcua-v2-ci run.

• v2-integration.yml (push to master, path-filtered PRs, manual; timeout-minutes: 60). integration-fixtures.sh up starts throwaway SQL Server 2022, PostgreSQL 17 (seeded from seed-pg.sql), QuestDB and opc-plc — plus a port-NAT opc-plc (otopcua-ci-opcplc-nat, host 50110 → 50000, private .invalid advertised host, OPCUA_NAT_ENDPOINT, #813) — (otopcua-ci-*, non-default host ports), exports the env vars the fixtures read, and down removes them. It uses plain docker run, not services: (act_runner support is undocumented and Postgres needs a seed step); the only runner requirement is a docker daemon. QuestDB is a throwaway container by decision (#841), not the shared /opt/otopcua-questdb fixture (#822): CI stays hermetic — no dependency on the 10.100.0.35 box, no state left by a developer's runs, nothing a concurrent CI job can share. What must not differ is the engine, so integration-fixtures.sh reads the image tag from the shared fixture's compose file (tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/Docker/docker-compose.yml, currently questdb/questdb:10.0.1, the same pin as docker-dev/docker-compose.questdb.yml) instead of hard-coding one, and fails up if that line is missing. Upgrade QuestDB by bumping the compose file (and the docker-dev overlay); CI_QDB_IMAGE overrides the image for a one-off trial. Both run QuestDB's defaults (incl. cairo.ttl.use.wall.clock=true, which the historian relies on). Runs the Sql, Historian.SqlServer, Historian.QuestDb, OpcUaClient and OpcUaClient.Browser *.IntegrationTests with MIN_EXECUTED=440. Local run: 468 executed / 8 skipped (Oracle x2, the blackhole docker pause gate, one QuestDB stress test, 4 others), about 4.5 min of test time, QuestDB the longest (~3.3 min). Not covered (no pullable image or needs credentials): Modbus/S7/AB/FOCAS/TwinCAT sims, Mqtt, MTConnect, Oracle, Host real-infra mode. #816/#818 added 8 executing tests here, floor NOT yet raised: ServerHostNameValidationTests (4, vs the plain otopcua-ci-opcplc) and the NAT None-browse / positive-secure / host-name cases (4; the certificate-trusting three run because integration-fixtures.sh exports the throwaway NAT container's public certificate as OPCUA_NAT_SERVER_CERT). Locally (2026-11-11, same CI filter) the two OpcUaClient projects went 13 → 21 executed with 0 skipped (Driver.OpcUaClient 9, Driver.OpcUaClient.Browser 12), and the NAT class passed 7/7 against a CI-style throwaway docker run of the NAT container plus the docker exec certificate hand-off. Once a hosted run's TRX shows Driver.OpcUaClient.trx executed=9 and Driver.OpcUaClient.Browser.trx executed=12, raise MIN_EXECUTED in v2-integration.yml 440 → 448 (and v2-e2e.yml 35 → 39). Hosted runs have not yet reached the test step (fixture start times out on the first network probe, #860), so this is still open (#818). If the NAT three show as skipped there, the certificate hand-off (docker exec … /app/pki/own/certs/*.der) did not work on the runner — fix that, do not lower the bump.
• v2-e2e.yml (manual + weekly cron only; timeout-minutes: 90). probe-e2e-fixtures.sh TCP-probes 10.100.0.35 ports 5020/1102/50000; if any is closed every later step is skipped and the run is green with a notice. Otherwise runs the Modbus, S7, OpcUaClient and OpcUaClient.Browser integration projects against the shared fleet with MIN_EXECUTED=35 (OPCUA_NAT_ENDPOINT points at the shared NAT stack on :50010; its 7 tests skip if it is down, so it is not a probe-required port — and its 3 certificate-trusting ones also skip if the stack's certificate no longer matches the committed Docker/nat/opc-plc-nat-server.der, so do not count them in the floor; #816's 4 ServerHostNameValidationTests need only the probe-required :50000, so this floor may go 35 → 39 after a hosted run shows them executed) (local: 42 executed / 38 skipped; Modbus skips the profiles that are not up). No test carries Category=E2E today, so the zero-match guard is this executed floor.