lmxopcua/scripts/ci/README.md

# Whole-solution CI

The `v2-ci` workflow restores, builds and tests in one job so the test step uses
that job's Release binaries. Restore disables parallel package work; restore and
build use at most two MSBuild workers. Testing uses one MSBuild worker, disables
parallel target-framework runs and loads `ci.runsettings`: one VSTest process and
two xUnit collection workers with conservative scheduling. Assemblies that disable
parallelization keep that setting.

The workflow retains the E2E/LiveIntegration exclusions, includes SlowUnit, keeps
Akka's test time factor at three, and requires at least 12,000 executed tests.
Runner capacity and TRX files remain available for diagnosing failures. These
limits reduce contention; they do not prove every timing failure has one cause.
There are no automatic test retries or changed assertion deadlines.

Gitea 1.26 supports the repository-wide `lmxopcua-v2-ci` concurrency group. Only
one run is active; an active run is not cancelled, but a newer pending run may
supersede an older pending run. The group does not serialize other repositories
using the same host.

Stall guards (#718): the job has `timeout-minutes: 120` (healthy runs take about 60 minutes)
and the test step passes `--blame-hang --blame-hang-timeout 20m --blame-hang-dump-type none`, so a
test that stops making progress is killed and named in the log instead of leaving hours of silence
(run 959 stalled 3h12m until the server reaper failed it).

To reproduce the bounded test leg after restore and Release build:

```bash
AKKA_TEST_TIMEFACTOR=3 dotnet test ZB.MOM.WW.OtOpcUa.slnx \
  --configuration Release --no-build -m:1 -p:TestTfmsInParallel=false \
  --settings scripts/ci/ci.runsettings \
  --filter "Category!=E2E&Category!=LiveIntegration" \
  --logger trx --results-directory artifacts/trx
MIN_EXECUTED=12000 bash scripts/ci/assert-not-all-skipped.sh artifacts/trx/*.trx
```

The guard uses the existing TRX `Counters/@executed` value for the aggregate
floor. Its `skipped` diagnostic counts individual `Results/UnitTestResult`
records whose outcome is `NotExecuted`, using the document's XML namespace.
Some VSTest reports set summary `notExecuted` to zero even when these records
are present. Skip records are counted per file, including repeated runs; they
are not a count of unique test methods and do not affect the executed floor.
For malformed XML, the diagnostic retains the old summary-counter fallback;
this guard does not add XML validation to its existing gate policy.

Run the deterministic script fixtures without building or running the solution:

```bash
python3 scripts/ci/test-assert-not-all-skipped.py
```

Verified hosted baseline: [Gitea run 975](https://gitea.dohertylan.com/dohertj2/lmxopcua/actions/runs/975)
completed successfully on 2026-10-13 for commit
`76bdca90e94e814e06ef10b78271e2b492b2ba32` (merged by PR #711).
Restore, Release build, README drift check, the serialized whole-solution test
step, the 12,000-test floor and TRX upload all passed. The floor counted
14,630 executed tests across 62 TRX files. This baseline predates the individual
skip-reporting correction; its printed zero skip diagnostic is not skip evidence.

## Fixture-backed integration and guarded E2E workflows (#757)

**Both are UNVERIFIED on a hosted Gitea runner until their first run**; they were validated locally only
(YAML parse, the exact `dotnet test` commands and fixture script against local throwaway containers).
Neither touches `v2-ci.yml`; each has its own concurrency group (`lmxopcua-v2-integration`,
`lmxopcua-v2-e2e`, `cancel-in-progress: false`) so a pending run of one cannot supersede a pending
`lmxopcua-v2-ci` run.

- **`v2-integration.yml`** (push to `master`, path-filtered PRs, manual; `timeout-minutes: 60`).
  `integration-fixtures.sh up` starts throwaway SQL Server 2022, PostgreSQL 17 (seeded from `seed-pg.sql`),
  QuestDB and opc-plc — plus a port-NAT opc-plc (`otopcua-ci-opcplc-nat`, host 50110 → 50000, private `.invalid`
  advertised host, `OPCUA_NAT_ENDPOINT`, #813) — (`otopcua-ci-*-<run id>`, non-default host ports; a port already taken on the shared runner host is retried one higher and the chosen ports are exported), exports the env vars the fixtures read, and
  `down` removes them. It uses plain `docker run`, not `services:` (act_runner support is undocumented and
  Postgres needs a seed step); the only runner requirement is a docker daemon.
  **QuestDB is a throwaway container by decision (#841), not the shared `/opt/otopcua-questdb` fixture (#822):** CI stays
  hermetic — no dependency on the 10.100.0.35 box, no state left by a developer's runs, nothing a concurrent CI job can
  share. What must not differ is the engine, so `integration-fixtures.sh` reads the image tag from the shared fixture's
  compose file (`tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/Docker/docker-compose.yml`,
  currently `questdb/questdb:10.0.1`, the same pin as `docker-dev/docker-compose.questdb.yml`) instead of hard-coding
  one, and fails `up` if that line is missing. Upgrade QuestDB by bumping the compose file (and the docker-dev overlay);
  `CI_QDB_IMAGE` overrides the image for a one-off trial. Both run QuestDB's defaults (incl.
  `cairo.ttl.use.wall.clock=true`, which the historian relies on). Runs the Sql, Historian.SqlServer,
  Historian.QuestDb, OpcUaClient and OpcUaClient.Browser `*.IntegrationTests` with `MIN_EXECUTED=448`.
  Local run: 468 executed / 8 skipped (Oracle x2, the blackhole `docker pause` gate, one QuestDB stress test,
  4 others), about 4.5 min of test time, QuestDB the longest (~3.3 min). Not covered (no pullable image or
  needs credentials): Modbus/S7/AB/FOCAS/TwinCAT sims, Mqtt, MTConnect, Oracle, Host real-infra mode.
  **#816/#818 added 8 executing tests here, floor NOT yet raised:** `ServerHostNameValidationTests` (4, vs the plain
  `otopcua-ci-opcplc`) and the NAT None-browse / positive-secure / host-name cases (4; the certificate-trusting three
  run because `integration-fixtures.sh` exports the throwaway NAT container's public certificate as
  `OPCUA_NAT_SERVER_CERT`). Locally (2026-11-11, same CI filter) the two OpcUaClient projects went 13 → 21 executed
  with 0 skipped (`Driver.OpcUaClient` 9, `Driver.OpcUaClient.Browser` 12), and the NAT class passed 7/7 against a
  CI-style throwaway `docker run` of the NAT container plus the `docker exec` certificate hand-off.
  **Hosted-confirmed 2026-11-11 (run 1225):** `Driver.OpcUaClient.trx` executed=9, `Driver.OpcUaClient.Browser.trx`
  executed=12, 525 executed in all, so `MIN_EXECUTED` in `v2-integration.yml` is now **448** (#818). The `v2-e2e.yml`
  35 → 39 raise still waits for a hosted e2e run.
  If the NAT three show as skipped there, the certificate
  hand-off (`docker exec … /app/pki/own/certs/*.der`) did not work on the runner — fix that, do not lower the bump.
- **`v2-e2e.yml`** (manual + weekly cron only; `timeout-minutes: 90`). `probe-e2e-fixtures.sh` TCP-probes
  10.100.0.35 ports 5020/1102/50000; if any is closed every later step is skipped and the run is green with a
  notice. Otherwise runs the Modbus, S7, OpcUaClient and OpcUaClient.Browser integration projects against the
  shared fleet with `MIN_EXECUTED=35` (`OPCUA_NAT_ENDPOINT` points at the shared NAT stack on :50010; its 7 tests
  skip if it is down, so it is not a probe-required port — and its 3 certificate-trusting ones also skip if the stack's
  certificate no longer matches the committed `Docker/nat/opc-plc-nat-server.der`, so do not count them in the floor;
  #816's 4 `ServerHostNameValidationTests` need only the probe-required :50000, so this floor may go **35 → 39** after
  a hosted run shows them executed) (local: 42 executed / 38 skipped; Modbus skips the profiles that are not
  up). No test carries `Category=E2E` today, so the zero-match guard is this executed floor.