Merge fix/812-endpoint-rewrite-family

This commit is contained in:
Joseph Doherty
2026-11-10 23:24:27 -05:00
17 changed files with 313 additions and 49 deletions
+3
View File
@@ -49,6 +49,9 @@ jobs:
env:
E2E_PROJECTS: >-
Driver.Modbus Driver.S7 Driver.OpcUaClient Driver.OpcUaClient.Browser
# #813: the port-NAT opc-plc stack (/opt/otopcua-opcuaclient-nat). Not in the probe's required ports —
# NatPublishedEndpointBrowseTests TCP-probe it themselves and skip if it is down.
OPCUA_NAT_ENDPOINT: opc.tcp://10.100.0.35:50010
run: |
set -euo pipefail
for p in $E2E_PROJECTS; do
+3 -2
View File
@@ -4,7 +4,8 @@
# it gets no slower or flakier. This workflow builds ONLY the five fixture-backed *.IntegrationTests
# projects and runs them against THROWAWAY containers that the job itself creates and removes
# (scripts/ci/integration-fixtures.sh): SQL Server 2022, PostgreSQL 17 (seeded), QuestDB and the public
# opc-plc image. Nothing here touches the shared 10.100.0.35 host, so CI does not depend on that box.
# opc-plc image (twice: plain, and a port-NAT copy behind OPCUA_NAT_ENDPOINT for the #806/#813 NAT browse tests).
# Nothing here touches the shared 10.100.0.35 host, so CI does not depend on that box.
#
# Explicit `docker run` steps rather than a `services:` block: whether the Gitea act_runner honours
# `services:` (and makes service ports reachable from the job) is not documented in infra/README.md §10,
@@ -79,7 +80,7 @@ jobs:
if: failure()
continue-on-error: true
run: |
for c in otopcua-ci-mssql otopcua-ci-pg otopcua-ci-questdb otopcua-ci-opcplc; do
for c in otopcua-ci-mssql otopcua-ci-pg otopcua-ci-questdb otopcua-ci-opcplc otopcua-ci-opcplc-nat; do
echo "=== $c ==="; docker logs --tail 40 "$c" 2>&1 || true
done
- name: stop fixtures
+1 -1
View File
@@ -276,7 +276,7 @@ central SQL Server.
> **Migrated 2026-04-28**: Docker config + host moved off this dev VM (DESKTOP-6JL3KKO) onto the shared Linux Docker host (`DOCKER`, 10.100.0.35) so the dev VM could shed WSL2/Hyper-V and have its GPU re-attached via ESXi passthrough. Docker Desktop is no longer installed here. All checked-in `appsettings.json` defaults, fixture-class default endpoints, and `e2e-config.sample.json` were rewritten to target `10.100.0.35`. See `docs/v2/dev-environment.md` for the full rewrite (header dated 2026-04-28).
> ⚠️ **The `project: lmxopcua` label is aspirational, not universal.** This note used to claim every fixture compose file carries it. As of 2026-07-24 only the **MQTT** fixture actually does — `grep -rn "labels:" tests --include=*.yml` matches `Driver.Mqtt.IntegrationTests/Docker/docker-compose.yml` and nothing else. So `docker ps --filter label=project=lmxopcua` returns the MQTT fixture alone, not the fleet. Add the label when you touch an older fixture; until then enumerate by container name.
> ⚠️ **The `project: lmxopcua` label is aspirational, not universal.** This note used to claim every fixture compose file carries it. As of 2026-07-24 only the **MQTT** fixture actually does (joined 2026-11-10 by the OpcUaClient port-NAT fixture, `Docker/nat/`, #813) — `grep -rn "labels:" tests --include=*.yml` matches `Driver.Mqtt.IntegrationTests/Docker/docker-compose.yml` and nothing else. So `docker ps --filter label=project=lmxopcua` returns the MQTT fixture alone, not the fleet. Add the label when you touch an older fixture; until then enumerate by container name.
Docker workloads run on a shared Linux host at **`10.100.0.35`** — not on this VM. Stacks live at `/opt/otopcua-<driver>/` on the host and carry the `project=lmxopcua` label so they're discoverable via `docker ps --filter label=project=lmxopcua`.
+5 -1
View File
@@ -60,7 +60,11 @@ otopcua-cli write -u opc.tcp://localhost:4840 -n "ns=2;s=MyNode" -v 42 -U operat
**Mapped ports and internal hostnames.** Endpoint discovery rewrites the host *and port* of the endpoint the server
advertises to the URL you passed (when the scheme matches), so connecting through a port mapping works even though the
server advertises its internal address (for example the docker-dev rig: `-u opc.tcp://localhost:4841` reaches the
node mapped to 4841, although every container advertises `:4840`).
node mapped to 4841, although every container advertises `:4840`). The advertised path is kept, a URL typed without
a port keeps the advertised port, and only the address changes — the security mode you asked for, the policy choice
and certificate validation are unaffected. Since #812 this is the same rewrite (`OpcUaEndpointUrlRewriter`) the
OpcUaClient driver and the AdminUI browse picker use; see `docs/drivers/OpcUaClient.md` §"Endpoint address". Unlike
those, the CLI fails loudly if the server advertises an endpoint with no or an unparseable URL.
When `-F` is provided, the shared service tries the primary URL first, then each failover URL in order. For long-running commands (`subscribe`, `alarms`), the service monitors the session via keep-alive and automatically reconnects to the next available server on failure.
+55 -6
View File
@@ -84,15 +84,62 @@ factory, the Test-Connect probe and the browse picker (unknown keys are skipped)
`ProbeTimeoutSeconds` (default 15, AdminUI "Probe timeout") caps the Test-Connect probe's deadline — it can shorten
the budget the host hands the probe, never extend it (review 2026-10-21 O-4; it used to be authored and consumed by nothing).
### Endpoint address: the configured host and port win (#676)
### Endpoint address: the configured host and port win (#676, #806, #812)
GetEndpoints returns each endpoint with the URL the **server** believes it lives at. Behind NAT, a port map, a
container port publish or a proxy that address is unreachable from the client — or reachable but bypassing the proxy.
The driver therefore rewrites the selected endpoint's **host and port** to the configured URL's whenever the schemes
match, keeping the advertised path (the SDK's own `CoreClientUtils.SelectEndpoint` rule, which the driver's
policy-aware endpoint selection used to bypass). A configured URL with **no port** keeps the advertised port (copying
the port-less `-1` would have sent the session to 4840). A different scheme is left alone; an unparseable advertised
URL falls back to the configured one. `OpcUaClientDriver.RewriteEndpointUrl`.
The selected endpoint's URL is therefore rewritten before the session opens. **One rule, one implementation** —
`OpcUaEndpointUrlRewriter.Rewrite` in `Driver.OpcUaClient.Contracts` (dependency-light: no OPC UA SDK) — used by all
three OPC UA client surfaces in this repo:
| Surface | Call site |
|---|---|
| Runtime driver | `OpcUaEndpointSelection.SelectMatching` (`Driver.OpcUaClient.Session`) via `OpcUaClientDriver` (#676) |
| AdminUI `/raw` live-browse picker | the same `SelectMatching` via `OpcUaClientDriverBrowser.SelectBrowseEndpoint` (#806) — so the picker connects exactly where the deployed driver will |
| Client CLI / UI | `EndpointSelector.SelectBest` (`Client.Shared`) (#812 — it used to carry its own copy) |
The rule, for `opc.tcp` (and any scheme, as long as advertised and configured schemes **match**):
- the **configured host and port** replace the advertised ones and the **advertised path is kept** (the server's to
choose; several endpoints may share a host:port) — the SDK's own `CoreClientUtils.SelectEndpoint` rule;
- a configured URL with **no port** keeps the advertised port (copying the port-less `-1` would have sent the
session to 4840);
- when host (case-insensitively — DNS is) and port already match, the advertised string is returned **verbatim**;
a real rewrite keeps the advertised path's percent-escaping (`AbsoluteUri`, #812);
- a **different scheme** (an `https` endpoint next to an `opc.tcp` one) is a different transport and is left alone;
- an unusable advertised URL (null / blank / unparseable) falls back to the configured one in the driver and the
browser; the Client CLI/UI instead reports it (it validates before calling the rewriter).
**What the rewrite does NOT touch.** It only chooses the network address. The **security policy and mode** are
still selected from the server's advertisement exactly as before — the driver and browser filter on the configured
`(SecurityPolicy, SecurityMode)` pair and fail loudly when nothing matches (the CLI/UI keeps its prefer-
`Basic256Sha256`-within-the-requested-mode choice) — and the selected endpoint's **server certificate and
user-token policies** are the advertised ones. **Certificate validation is unchanged**: the secure channel still
validates the certificate of the server actually reached, against the same trust store (the driver's PKI, the
browse picker's separate `adminui-browse-pki`, the CLI's own) with the same `AutoAcceptCertificates` semantics. The
NAT tests below pin that a secure browse through the NAT is refused on an untrusted certificate and succeeds once it
is trusted.
**Running the NAT tests.** `NatPublishedEndpointBrowseTests`
(`tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.OpcUaClient.Browser.IntegrationTests`, `Fixture=opc-plc-nat`) run against
an opc-plc whose advertised host is private (`*.invalid`) **and** whose published port differs from its internal
one; each test first asserts that premise, so a fixture advertising a reachable address cannot pass them vacuously.
They skip unless `OPCUA_NAT_ENDPOINT` is set and accepts TCP. The shared-host fixture (#813, stack
`/opt/otopcua-opcuaclient-nat`, compose in `tests/.../Driver.OpcUaClient.IntegrationTests/Docker/nat/`):
```bash
ssh dohertj2@10.100.0.35 'cd /opt/otopcua-opcuaclient-nat && docker compose up -d'
OPCUA_NAT_ENDPOINT=opc.tcp://10.100.0.35:50010 dotnet test \
tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.OpcUaClient.Browser.IntegrationTests --filter Fixture=opc-plc-nat
```
The secure case asserts a certificate **refusal** by default. To make it positive, trust the fixture's server
certificate in the browse PKI and set `OPCUA_NAT_SERVER_CERT_TRUSTED=1` — the certificate is regenerated whenever
the container is recreated, so fetch the current one:
`ssh dohertj2@10.100.0.35 'docker exec otopcua-opc-plc-nat sh -c "cat /app/pki/own/certs/*.der"' > "<LocalAppData>/OtOpcUa/adminui-browse-pki/trusted/certs/opc-plc-nat.der"`
(`~/Library/Application Support/...` on macOS, `~/.local/share/...` on Linux), and delete it again afterwards. CI
(`v2-integration.yml`) starts its own throwaway NAT copy (`scripts/ci/integration-fixtures.sh`); `v2-e2e.yml` points at
the shared one. Neither trusts the certificate, so CI runs the refusal form.
### Session lifetime
@@ -396,6 +443,8 @@ They are not part of `DriverHealth` or the
- **Integration fixture** — exercises the driver against a reference OPC UA
server (opc-plc) on the shared docker host; see
[OpcUaClient-Test-Fixture.md](OpcUaClient-Test-Fixture.md) for the coverage map.
- **NAT / port-publish browse** — `NatPublishedEndpointBrowseTests` against the port-NAT opc-plc
(`OPCUA_NAT_ENDPOINT`); see "Running the NAT tests" under the endpoint-address section above.
- **Wire tests against the in-process server** — `tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests/`
`OpcUaClientDriver*WireTests` (no Docker). `OpcUaClientDriverStallWireTests` (#676) covers the in-place blip that
used to be live-gate-only: the driver connects through `TcpStallProxy`, an in-test TCP forwarder whose pause holds
@@ -142,6 +142,15 @@ Razor picker body BrowserSessionService IDriverBrowser
- Builds a **separate** `ApplicationConfiguration` from the runtime driver — PKI root at `%LocalAppData%/OtOpcUa/adminui-browse-pki/` (separate cert store)
- `ApplicationName = "OtOpcUa AdminUI Browse"`, `ApplicationUri = "urn:OtOpcUa:AdminUI:Browse"`
- Endpoint selection: same `DiscoveryClient.GetEndpointsAsync` → filter `(policy, mode)` as the runtime driver
— literally the same code since #806: `OpcUaEndpointSelection.SelectMatching` (`Driver.OpcUaClient.Session`), which
then rewrites the chosen endpoint's URL through the shared `OpcUaEndpointUrlRewriter` (`Driver.OpcUaClient.Contracts`;
also used by the Client CLI/UI since #812). For `opc.tcp`, the **operator-configured host and port** replace the
advertised ones and the **advertised path is kept** (a port-less configured URL keeps the advertised port; a different
scheme is left alone). Before #806 the browser dialled the advertised URL verbatim, so behind NAT / a container port
publish it failed DNS on the server's private hostname where the deployed driver connected fine.
Only the address changes: policy + mode selection, the advertised server certificate and user-token policies, and
certificate validation against the browse PKI are exactly as before. Full rule: `docs/drivers/OpcUaClient.md`
§"Endpoint address".
- One endpoint only (no failover) — interactive use; user retries with different URL on failure
- Bounded by `OpcUaClientDriverOptions.PerEndpointConnectTimeout` (clamped [5, 30]s)
@@ -265,6 +274,14 @@ Razor picker body BrowserSessionService IDriverBrowser
### Integration tests (opt-in, fixture-gated)
- `tests/Drivers/.../OpcUaClient.Browser.IntegrationTests/`: end-to-end against opc-plc, 3-level expand + round-trip resolve. Skipped unless `OPCUA_SIM_ENDPOINT` set
- Same project, `NatPublishedEndpointBrowseTests` (`Fixture=opc-plc-nat`, #806): browse through a port NAT — an opc-plc
advertising a private `.invalid` host and an internal port different from the published one. Skipped unless
`OPCUA_NAT_ENDPOINT` is set; the shared-host fixture (#813) is `opc.tcp://10.100.0.35:50010`
(`ssh dohertj2@10.100.0.35 'cd /opt/otopcua-opcuaclient-nat && docker compose up -d'`, compose in
`tests/.../Driver.OpcUaClient.IntegrationTests/Docker/nat/`), and CI starts a throwaway copy. Run:
`OPCUA_NAT_ENDPOINT=opc.tcp://10.100.0.35:50010 dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.OpcUaClient.Browser.IntegrationTests --filter Fixture=opc-plc-nat`.
The secure case asserts a certificate refusal unless the fixture cert is trusted in the browse PKI and
`OPCUA_NAT_SERVER_CERT_TRUSTED=1` (see `docs/drivers/OpcUaClient.md`).
- No Galaxy integration suite in v1 (requires wonder-app-vd03; deferred)
### Specific regression tests
+18 -5
View File
@@ -37,7 +37,7 @@ ssh dohertj2@10.100.0.35 'docker ps --filter label=project=lmxopcua --format "{{
| Stack dir | Purpose |
|---|---|
| `/opt/otopcua-mssql` | central SQL (always-on) |
| `/opt/otopcua-modbus` · `/opt/otopcua-abcip` · `/opt/otopcua-s7` · `/opt/otopcua-opcuaclient` · `/opt/otopcua-mqtt` · `/opt/otopcua-mtconnect` · `/opt/otopcua-focas` | driver fixtures |
| `/opt/otopcua-modbus` · `/opt/otopcua-abcip` · `/opt/otopcua-s7` · `/opt/otopcua-opcuaclient` · `/opt/otopcua-opcuaclient-nat` · `/opt/otopcua-mqtt` · `/opt/otopcua-mtconnect` · `/opt/otopcua-focas` | driver fixtures |
| `~/otopcua-ablegacy` | driver fixture (user-owned) |
| `~/otopcua-harness` | Host.IntegrationTests real-mode deps (SQL + GLAuth) — see §4 |
@@ -68,13 +68,14 @@ when it isn't running. Bring one up, `dotnet test`, tear down. Repo compose live
| AB Legacy (PCCC) | `otopcua-ab-server` (PCCC mode) | `10.100.0.35:44818` | `AB_LEGACY_COMPOSE_PROFILE=<…> docker compose up -d` (stack `~/otopcua-ablegacy`) |
| S7 | `otopcua-python-snap7` | `10.100.0.35:1102` | `docker compose --profile s7_1500 up -d` |
| OpcUaClient | `mcr.microsoft.com/iotedge/opc-plc:2.14.10` | `opc.tcp://10.100.0.35:50000` | `docker compose up -d` |
| OpcUaClient — port NAT (#813) | same opc-plc image; advertises `opc.tcp://otopcua-opc-plc-nat-private.invalid:50000` | `opc.tcp://10.100.0.35:50010` (`OPCUA_NAT_ENDPOINT`, no default) | `docker compose up -d` (stack `/opt/otopcua-opcuaclient-nat`; rsync the repo's OpcUaClient `Docker/nat/` dir first) |
| FOCAS | `otopcua-focas-fixture` (vendored focas-mock, built on the host) | `10.100.0.35:8193` (published) | `docker compose up -d --build` (stack `/opt/otopcua-focas`; rsync the repo's FOCAS `Docker/` dir first) |
| MQTT | `eclipse-mosquitto:2.0.22` (+ a publisher sidecar) | `10.100.0.35:8883` TLS+auth · `:1883` plaintext+auth | `MQTT_FIXTURE_USERNAME=otopcua MQTT_FIXTURE_PASSWORD=<pw> docker compose up -d` (stack `/opt/otopcua-mqtt`) |
| MQTT — Sparkplug B | `otopcua-sparkplug-sim` (project-owned C# edge-node simulator) | same broker; group **`OtOpcUaSim`**, edge nodes **`EdgeA`** / **`EdgeB`**, `EdgeA` device **`Filler1`**, ~2 s cadence | `docker compose --profile sparkplug up -d` (same stack). Answers rebirth NCMDs; restart it to force a fresh birth |
| MTConnect | `mtconnect/agent:2.7.0.12` + `python:3.13-alpine` adapter | `http://10.100.0.35:5000` | `docker compose up -d --wait` |
**Endpoint overrides** point a suite at a real PLC instead of the sim:
`MODBUS_SIM_ENDPOINT` · `AB_SERVER_ENDPOINT` · `S7_SIM_ENDPOINT` · `OPCUA_SIM_ENDPOINT` ·
`MODBUS_SIM_ENDPOINT` · `AB_SERVER_ENDPOINT` · `S7_SIM_ENDPOINT` · `OPCUA_SIM_ENDPOINT` · `OPCUA_NAT_ENDPOINT` (opt-in, no default) ·
`MQTT_FIXTURE_ENDPOINT` / `MQTT_FIXTURE_PLAIN_ENDPOINT` · `MTCONNECT_AGENT_ENDPOINT` · `OTOPCUA_FOCAS_SIM_ENDPOINT`.
> **`otopcua-focas-sim` on the host is NOT ours (#651).** It is ignitionoee's mock (created by that
@@ -91,7 +92,17 @@ when it isn't running. Bring one up, `dotnet test`, tear down. Repo compose live
> CA + server cert/key, gitignored) and `MQTT_FIXTURE_USERNAME`/`MQTT_FIXTURE_PASSWORD` are exported
> at `docker compose up`. The live suite additionally wants `MQTT_FIXTURE_CA_CERT` pointing at a local
> copy of the CA: `scp dohertj2@10.100.0.35:/opt/otopcua-mqtt/secrets/ca.crt /tmp/mqtt-fixture-ca.crt`.
> It is also the only fixture whose services carry the `project: lmxopcua` label (see CLAUDE.md).
> It and the OpcUaClient NAT fixture are the only ones whose compose files carry the `project: lmxopcua` label (see CLAUDE.md).
> **The OpcUaClient NAT fixture exists for what it ADVERTISES (#806/#813).** `--ph` sets a private `.invalid`
> host (RFC 6761: never resolves, so no DNS search domain can make it reachable) and the published port 50010 differs
> from the internal 50000, so every GetEndpoints URL is unreachable on both host and port. Only
> `NatPublishedEndpointBrowseTests` (Browser.IntegrationTests, `Fixture=opc-plc-nat`) use it, and each asserts that
> premise before testing the shared `OpcUaEndpointUrlRewriter` (configured host:port win, advertised path kept):
> `OPCUA_NAT_ENDPOINT=opc.tcp://10.100.0.35:50010 dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.OpcUaClient.Browser.IntegrationTests --filter Fixture=opc-plc-nat`.
> A separate stack so it never touches `otopcua-opc-plc`; it carries the `project: lmxopcua` label. The secure case
> turns positive with `OPCUA_NAT_SERVER_CERT_TRUSTED=1` once the container's cert is in the browse PKI
> (`docs/drivers/OpcUaClient.md` §"Endpoint address"). CI runs its own throwaway copy (`otopcua-ci-opcplc-nat`, §10).
> **MTConnect is a TWO-service stack.** The `mtconnect/agent` image ships no simulator, so a lone
> Agent answers `/probe` and reports every observation `UNAVAILABLE` forever. The `adapter` service
@@ -261,6 +272,8 @@ Things that are not obvious (#606):
`v2-integration.yml` runs the fixture-backed Sql / Historian (SQL Server, QuestDB) / OpcUaClient integration
projects against throwaway containers the job starts itself (`scripts/ci/integration-fixtures.sh`; not the
shared 10.100.0.35 host), floor `MIN_EXECUTED=440`. `v2-e2e.yml` is manual/weekly and skips cleanly unless
10.100.0.35 is reachable from the runner. Each has its own concurrency group; see
shared 10.100.0.35 host), floor `MIN_EXECUTED=440`; that includes a port-NAT opc-plc (`otopcua-ci-opcplc-nat`,
host port 50110, exported as `OPCUA_NAT_ENDPOINT`) for the NAT browse tests (#813). `v2-e2e.yml` is manual/weekly and
skips cleanly unless 10.100.0.35 is reachable from the runner; it sets `OPCUA_NAT_ENDPOINT=opc.tcp://10.100.0.35:50010`
(the shared NAT stack, not a probe-required port — its tests skip if it is down). Each has its own concurrency group; see
[`scripts/ci/README.md`](../scripts/ci/README.md).
+4 -2
View File
@@ -67,7 +67,8 @@ Neither touches `v2-ci.yml`; each has its own concurrency group (`lmxopcua-v2-in
- **`v2-integration.yml`** (push to `master`, path-filtered PRs, manual; `timeout-minutes: 60`).
`integration-fixtures.sh up` starts throwaway SQL Server 2022, PostgreSQL 17 (seeded from `seed-pg.sql`),
QuestDB and opc-plc (`otopcua-ci-*`, non-default host ports), exports the env vars the fixtures read, and
QuestDB and opc-plc — plus a port-NAT opc-plc (`otopcua-ci-opcplc-nat`, host 50110 → 50000, private `.invalid`
advertised host, `OPCUA_NAT_ENDPOINT`, #813) — (`otopcua-ci-*`, non-default host ports), exports the env vars the fixtures read, and
`down` removes them. It uses plain `docker run`, not `services:` (act_runner support is undocumented and
Postgres needs a seed step); the only runner requirement is a docker daemon. Runs the Sql, Historian.SqlServer,
Historian.QuestDb, OpcUaClient and OpcUaClient.Browser `*.IntegrationTests` with `MIN_EXECUTED=440`.
@@ -77,5 +78,6 @@ Neither touches `v2-ci.yml`; each has its own concurrency group (`lmxopcua-v2-in
- **`v2-e2e.yml`** (manual + weekly cron only; `timeout-minutes: 90`). `probe-e2e-fixtures.sh` TCP-probes
10.100.0.35 ports 5020/1102/50000; if any is closed every later step is skipped and the run is green with a
notice. Otherwise runs the Modbus, S7, OpcUaClient and OpcUaClient.Browser integration projects against the
shared fleet with `MIN_EXECUTED=35` (local: 42 executed / 38 skipped; Modbus skips the profiles that are not
shared fleet with `MIN_EXECUTED=35` (`OPCUA_NAT_ENDPOINT` points at the shared NAT stack on :50010; its 3 tests
skip if it is down, so it is not a probe-required port) (local: 42 executed / 38 skipped; Modbus skips the profiles that are not
up). No test carries `Category=E2E` today, so the zero-match guard is this executed floor.
+10 -2
View File
@@ -2,7 +2,7 @@
#
# integration-fixtures.sh — throwaway, self-contained fixtures for the v2-integration workflow (#757).
#
# up start SQL Server, PostgreSQL (seeded), QuestDB and opc-plc as containers named
# up start SQL Server, PostgreSQL (seeded), QuestDB, opc-plc and a port-NAT opc-plc as containers named
# otopcua-ci-*, wait until each is usable, and print the env exports the
# *.IntegrationTests fixtures read (also appended to $GITHUB_ENV when set).
# down remove those containers (idempotent; touches nothing else).
@@ -21,10 +21,13 @@ PG_PORT="${CI_PG_PORT:-15440}"
QDB_PORT="${CI_QDB_PORT:-18812}"
QDB_HTTP_PORT="${CI_QDB_HTTP_PORT:-19000}"
OPC_PORT="${CI_OPC_PORT:-50100}"
# #813 NAT opc-plc: published on OPC_NAT_PORT, listening on 50000 inside, advertising a private `.invalid`
# host — so its GetEndpoints URLs are unreachable on BOTH host and port (NatPublishedEndpointBrowseTests).
OPC_NAT_PORT="${CI_OPC_NAT_PORT:-50110}"
# Throwaway-container-only credentials (never used against anything else).
SA_PASSWORD="${CI_SA_PASSWORD:-Ci_throwaway_Pw1}"
PG_PASSWORD="${CI_PG_PASSWORD:-Ci_throwaway_pg1}"
NAMES=(otopcua-ci-mssql otopcua-ci-pg otopcua-ci-questdb otopcua-ci-opcplc)
NAMES=(otopcua-ci-mssql otopcua-ci-pg otopcua-ci-questdb otopcua-ci-opcplc otopcua-ci-opcplc-nat)
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
fixture_host() {
@@ -59,6 +62,9 @@ up() {
questdb/questdb:10.0.1 >/dev/null
docker run -d --name otopcua-ci-opcplc -p "$OPC_PORT:$OPC_PORT" \
mcr.microsoft.com/iotedge/opc-plc:2.14.10 --pn="$OPC_PORT" --ut --aa --alm >/dev/null
# Mirrors tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.OpcUaClient.IntegrationTests/Docker/nat/docker-compose.yml.
docker run -d --name otopcua-ci-opcplc-nat --hostname otopcua-ci-opcplc-nat-private -p "$OPC_NAT_PORT:50000" \
mcr.microsoft.com/iotedge/opc-plc:2.14.10 --pn=50000 --ph=otopcua-ci-opcplc-nat-private.invalid --ut --aa >/dev/null
wait_for "postgres" 120 docker exec otopcua-ci-pg pg_isready -U postgres -d sqlpollfixture
# pg_isready can answer during initdb's temp server; require a real query on the final server.
@@ -73,6 +79,7 @@ up() {
wait_for "sql server" 300 docker exec otopcua-ci-mssql "$sqlcmd" -C -S localhost -U sa -P "$SA_PASSWORD" -Q 'SELECT 1'
wait_for "questdb" 120 curl -fsS "http://$(fixture_host):$QDB_HTTP_PORT/exec?query=select%201"
wait_for "opc-plc" 120 bash -c "exec 3<>/dev/tcp/$(fixture_host)/$OPC_PORT"
wait_for "opc-plc (NAT)" 120 bash -c "exec 3<>/dev/tcp/$(fixture_host)/$OPC_NAT_PORT"
local h; h="$(fixture_host)"
local -a exports=(
@@ -84,6 +91,7 @@ up() {
"HISTORIAN_SQL_TEST_CONNECTION=Server=$h,$SQL_PORT;User Id=sa;Password=$SA_PASSWORD;TrustServerCertificate=True;Encrypt=False"
"HISTORIAN_QUESTDB_TEST_CONNECTION=Host=$h;Port=$QDB_PORT;Username=admin;Password=quest;Database=qdb;ServerCompatibilityMode=NoTypeLoading"
"OPCUA_SIM_ENDPOINT=opc.tcp://$h:$OPC_PORT"
"OPCUA_NAT_ENDPOINT=opc.tcp://$h:$OPC_NAT_PORT"
)
for e in "${exports[@]}"; do
echo "export \"$e\""
@@ -1,6 +1,7 @@
using Opc.Ua;
using Opc.Ua.Client;
using Serilog;
using ZB.MOM.WW.OtOpcUa.Driver.OpcUaClient;
namespace ZB.MOM.WW.OtOpcUa.Client.Shared.Adapters;
@@ -36,8 +37,7 @@ internal static class EndpointSelector
/// <summary>
/// Picks the best endpoint from the discovery response that matches the requested
/// security mode, preferring <c>Basic256Sha256</c>, and rewrites the endpoint URL
/// host to match the user-supplied URL when the discovery response advertises a
/// different hostname.
/// host and port to the user-supplied ones via the shared <see cref="OpcUaEndpointUrlRewriter"/>.
/// </summary>
/// <param name="allEndpoints">Endpoints returned by the discovery query, in any order.</param>
/// <param name="endpointUrl">The endpoint URL the operator supplied; supplies the hostname rewrite target.</param>
@@ -85,22 +85,23 @@ internal static class EndpointSelector
$"No endpoint found with security mode '{requestedMode}'. Available endpoints: {available}");
}
// Rewrite the endpoint URL host AND port to the user-supplied ones (as the SDK's
// CoreClientUtils.SelectEndpoint does). A server behind a port mapping or internal DNS
// advertises its own host:port (docker-dev containers all say :4840); following that would
// open the session on a different server and fail the secure-channel certificate check.
// Only when the scheme matches: a different scheme (https vs opc.tcp) is a different transport.
var serverUri = new Uri(best.EndpointUrl ?? throw new InvalidOperationException("Endpoint has no URL."));
var requestedUri = new Uri(endpointUrl);
// A URL typed without a port (opc.tcp has no registered default, so Port is -1) keeps the advertised port.
var port = requestedUri.Port > 0 ? requestedUri.Port : serverUri.Port;
if (string.Equals(serverUri.Scheme, requestedUri.Scheme, StringComparison.OrdinalIgnoreCase)
&& (serverUri.Host != requestedUri.Host || serverUri.Port != port))
// Rewrite the endpoint URL host AND port to the user-supplied ones, keeping the advertised path. A server
// behind a port mapping or internal DNS advertises its own host:port (docker-dev containers all say :4840);
// following that would open the session on a different server and fail the secure-channel certificate check.
// The rule is the ONE shared with the OpcUaClient driver and the AdminUI browse picker (#812, #806, #676):
// same scheme only, configured port only when it names one, the advertised string verbatim on a no-op.
//
// Validation stays here, not in the rewriter: the shared rewriter falls back to the configured URL on an
// unusable advertised one (right for a deployed driver), while this interactive tool reports it — a null URL
// throws InvalidOperationException and an unparseable one (either side) UriFormatException, as before #812.
var advertisedUrl = best.EndpointUrl ?? throw new InvalidOperationException("Endpoint has no URL.");
_ = new Uri(advertisedUrl);
_ = new Uri(endpointUrl);
var rewritten = OpcUaEndpointUrlRewriter.Rewrite(advertisedUrl, endpointUrl);
if (!string.Equals(rewritten, advertisedUrl, StringComparison.Ordinal))
{
var builder = new UriBuilder(best.EndpointUrl) { Host = requestedUri.Host, Port = port };
best.EndpointUrl = builder.ToString();
Logger.Debug("Rewrote endpoint from {ServerHost}:{ServerPort} to {RequestedHost}:{RequestedPort}",
serverUri.Host, serverUri.Port, requestedUri.Host, port);
best.EndpointUrl = rewritten;
Logger.Debug("Rewrote endpoint from {AdvertisedUrl} to {RewrittenUrl}", advertisedUrl, rewritten);
}
return best;
@@ -12,6 +12,14 @@
<PackageReference Include="Serilog"/>
</ItemGroup>
<ItemGroup>
<!-- #812: the endpoint-URL rewrite rule shared with the OpcUaClient driver + AdminUI browse picker
(OpcUaEndpointUrlRewriter). Contracts is dependency-light by contract (Core.Abstractions only, no
NuGet packages, no OPC UA SDK; pinned by ContractsIsDependencyLightTests), so this adds two small
first-party assemblies and nothing else to the CLI/UI. -->
<ProjectReference Include="..\..\Drivers\ZB.MOM.WW.OtOpcUa.Driver.OpcUaClient.Contracts\ZB.MOM.WW.OtOpcUa.Driver.OpcUaClient.Contracts.csproj"/>
</ItemGroup>
<ItemGroup>
<InternalsVisibleTo Include="ZB.MOM.WW.OtOpcUa.Client.Shared.Tests"/>
</ItemGroup>
@@ -5,7 +5,9 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.OpcUaClient;
/// OpcUaClient driver family. A server reports its GetEndpoints results with ITS view of its own address; behind
/// NAT, a port map, a container port publish or a proxy that address is unreachable from here (or bypasses the
/// proxy). Shared by the runtime driver (Gitea #676) and the AdminUI live-browse picker (Gitea #806, which used
/// the advertised URL unchanged and failed DNS on a published-port fixture the driver handled).
/// the advertised URL unchanged and failed DNS on a published-port fixture the driver handled), and — since
/// Gitea #812 — by the Client CLI/UI's <c>EndpointSelector</c> (Client.Shared), whose own copy of this rule it
/// replaced.
/// </summary>
/// <remarks>
/// Pure string/<see cref="Uri"/> logic, deliberately in <c>.Contracts</c>: it is not session-typed, so it must not
@@ -18,8 +20,10 @@ public static class OpcUaEndpointUrlRewriter
/// <summary>
/// The address to open the session on. When the advertised URL's scheme matches the configured one, the
/// configured host AND port (when it names one) replace the advertised ones and the advertised path is kept —
/// the SDK's own <c>CoreClientUtils.SelectEndpoint</c> rule. A different scheme is a different transport and is
/// left alone; an unparseable advertised URL falls back to the configured one.
/// the SDK's own <c>CoreClientUtils.SelectEndpoint</c> rule. When host (case-insensitively) and port already
/// match, the advertised string is returned verbatim. A different scheme is a different transport and is left
/// alone; an unparseable advertised URL falls back to the configured one (callers that must fail loudly
/// instead, like the Client CLI/UI, validate before calling).
/// </summary>
/// <param name="advertisedUrl">The <c>EndpointUrl</c> the server returned from GetEndpoints.</param>
/// <param name="configuredUrl">The URL the operator configured (the one GetEndpoints was sent to).</param>
@@ -32,12 +36,16 @@ public static class OpcUaEndpointUrlRewriter
return configuredUrl;
if (!string.Equals(advertised.Scheme, configured.Scheme, StringComparison.OrdinalIgnoreCase)) return advertisedUrl;
// A port-less configured URL (Uri.Port == -1 for opc.tcp) keeps the advertised port — copying -1 would drop it
// and the SDK would fall back to 4840 (review of 3b88056f; same fix as the client's EndpointSelector).
var builder = new UriBuilder(advertised)
{
Host = configured.DnsSafeHost,
Port = configured.Port > 0 ? configured.Port : advertised.Port,
};
return builder.Uri.ToString();
// and the SDK would fall back to 4840 (review of 3b88056f).
var port = configured.Port > 0 ? configured.Port : advertised.Port;
// Nothing to rewrite: return the advertised string verbatim rather than a re-serialised copy (no added
// trailing slash, no case-folding of a host that only differs in case — DNS is case-insensitive). This is the
// Client CLI/UI's long-standing rule, kept when its own copy was folded in here (#812).
if (string.Equals(advertised.DnsSafeHost, configured.DnsSafeHost, StringComparison.OrdinalIgnoreCase)
&& advertised.Port == port)
return advertisedUrl;
var builder = new UriBuilder(advertised) { Host = configured.DnsSafeHost, Port = port };
// AbsoluteUri, not ToString(): ToString() un-escapes, turning an advertised "/x%20y" into "/x y" (#812).
return builder.Uri.AbsoluteUri;
}
}
@@ -0,0 +1,77 @@
using Opc.Ua;
using Shouldly;
using Xunit;
using ZB.MOM.WW.OtOpcUa.Client.Shared.Adapters;
namespace ZB.MOM.WW.OtOpcUa.Client.Shared.Tests.Adapters;
/// <summary>
/// Gitea #812 — pins the Client CLI/UI endpoint-URL rewrite (<see cref="EndpointSelector.SelectBest"/>) case by case,
/// reusing the #806 rewrite cases of <c>OpcUaClientEndpointRewriteTests</c> (Driver.OpcUaClient.Tests). These were
/// written and run green against Client.Shared's own pre-#812 copy of the rule, BEFORE it was swapped for the shared
/// <c>OpcUaEndpointUrlRewriter</c>, so they are the "behaviour stays identical" proof for the swap.
/// </summary>
public sealed class EndpointSelectorRewriteParityTests
{
private static string Select(string? advertised, string configured)
{
var ep = new EndpointDescription
{
EndpointUrl = advertised, SecurityMode = MessageSecurityMode.None, SecurityPolicyUri = SecurityPolicies.None,
};
return EndpointSelector.SelectBest([ep], configured, MessageSecurityMode.None).EndpointUrl!;
}
[Theory]
// #806 cases (exact strings shared with the driver's rewriter tests).
[InlineData("opc.tcp://internal-host:4840/OtOpcUa", "opc.tcp://10.1.2.3:54840/OtOpcUa", "opc.tcp://10.1.2.3:54840/OtOpcUa")]
[InlineData("opc.tcp://otopcua-sdk1-browser-nat-parity:50000/", "opc.tcp://127.0.0.1:32772", "opc.tcp://127.0.0.1:32772/")]
[InlineData("opc.tcp://plc-a:4840/Server/Ua", "opc.tcp://gateway:6000/", "opc.tcp://gateway:6000/Server/Ua")]
[InlineData("opc.tcp://127.0.0.1:4840/OtOpcUa", "opc.tcp://127.0.0.1:61000/OtOpcUa", "opc.tcp://127.0.0.1:61000/OtOpcUa")]
[InlineData("opc.tcp://plc-a:4840/Server", "opc.tcp://plc-a:4840/Server", "opc.tcp://plc-a:4840/Server")]
[InlineData("opc.tcp://internal-host:62541/Server", "opc.tcp://gateway/Server", "opc.tcp://gateway:62541/Server")]
[InlineData("opc.tcp://internal-host:4840/OtOpcUa", "opc.tcp://[fd00::1]:54840/OtOpcUa", "opc.tcp://[fd00::1]:54840/OtOpcUa")]
[InlineData("opc.tcp://[fd00::7]:4840/Ua", "opc.tcp://gateway:6000", "opc.tcp://gateway:6000/Ua")]
[InlineData("https://srv:443/ua", "opc.tcp://srv:4840/ua", "https://srv:443/ua")]
// The #813 NAT fixture's shape: private .invalid host AND a different internal port.
[InlineData("opc.tcp://otopcua-opc-plc-nat-private.invalid:50000", "opc.tcp://10.100.0.35:50010", "opc.tcp://10.100.0.35:50010/")]
// Client.Shared-specific: a no-op rewrite returns the advertised string VERBATIM (no trailing slash added) …
[InlineData("opc.tcp://server:4840", "opc.tcp://server:4840", "opc.tcp://server:4840")]
// … and a case-only host difference is not a rewrite (DNS is case-insensitive; the advertised spelling stays).
[InlineData("opc.tcp://PLC-A:4840/Server", "OPC.TCP://plc-a:4840/", "opc.tcp://PLC-A:4840/Server")]
[InlineData("opc.tcp://plc-a:4840/Server", "opc.tcp://PLC-A:4840/", "opc.tcp://plc-a:4840/Server")]
// A real rewrite keeps the advertised path's percent-escaping intact.
[InlineData("opc.tcp://a:4840/x%20y?q=1", "opc.tcp://b:1/", "opc.tcp://b:1/x%20y?q=1")]
// An IPv6 zone index on the configured host is dropped (UriBuilder behaviour, same in both copies).
[InlineData("opc.tcp://internal-host:4840/OtOpcUa", "opc.tcp://[fe80::1%25eth0]:4841", "opc.tcp://[fe80::1]:4841/OtOpcUa")]
public void SelectBest_rewrites_exactly_as_before(string advertised, string configured, string expected)
{
Select(advertised, configured).ShouldBe(expected);
}
/// <summary>
/// Client.Shared fails LOUDLY on an unusable URL, where the driver/browser rewriter falls back to the configured
/// URL. The CLI/UI is an interactive diagnostic tool talking to whatever the operator typed, so a server
/// advertising garbage is reported rather than papered over. Kept as call-site validation in #812.
/// </summary>
[Fact]
public void SelectBest_throws_on_an_endpoint_without_a_url()
{
Should.Throw<InvalidOperationException>(() => Select(null, "opc.tcp://srv:4840/ua"))
.Message.ShouldContain("no URL");
}
[Theory]
[InlineData("")]
[InlineData("not a url")]
public void SelectBest_throws_on_an_unparseable_advertised_url(string advertised)
{
Should.Throw<UriFormatException>(() => Select(advertised, "opc.tcp://srv:4840/ua"));
}
[Fact]
public void SelectBest_throws_on_an_unparseable_configured_url()
{
Should.Throw<UriFormatException>(() => Select("opc.tcp://a:4840/x", "not a url"));
}
}
@@ -14,8 +14,12 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.OpcUaClient.Browser.IntegrationTests;
/// whose internal port differs from the published one, so its GetEndpoints advertise an address this machine
/// cannot reach:
/// <code>
/// # shared docker host (#813): compose in Driver.OpcUaClient.IntegrationTests/Docker/nat/
/// ssh dohertj2@10.100.0.35 'cd /opt/otopcua-opcuaclient-nat &amp;&amp; docker compose up -d'
/// OPCUA_NAT_ENDPOINT=opc.tcp://10.100.0.35:50010 dotnet test --filter Fixture=opc-plc-nat
/// # or locally / in CI (scripts/ci/integration-fixtures.sh starts the same shape as otopcua-ci-opcplc-nat):
/// docker run -d --rm --name otopcua-806-nat --hostname otopcua-806-nat-parity -p 32781:50000 \
/// mcr.microsoft.com/iotedge/opc-plc:2.14.10 --pn=50000 --autoaccept --ut
/// mcr.microsoft.com/iotedge/opc-plc:2.14.10 --pn=50000 --ph=otopcua-806-nat-parity.invalid --autoaccept --ut
/// OPCUA_NAT_ENDPOINT=opc.tcp://127.0.0.1:32781 dotnet test --filter Fixture=opc-plc-nat
/// </code>
/// Skipped unless <c>OPCUA_NAT_ENDPOINT</c> is set and accepts TCP. Every test first proves the premise (the
@@ -10,6 +10,7 @@ here since opc-plc ships pre-containerized.
| File | Purpose |
|---|---|
| [`docker-compose.yml`](docker-compose.yml) | Service definition for `otopcua-opc-plc` — image pin, port map, command flags. |
| [`nat/docker-compose.yml`](nat/docker-compose.yml) | `otopcua-opc-plc-nat` — a SEPARATE stack (`/opt/otopcua-opcuaclient-nat` on the shared host): opc-plc advertising a private `.invalid` host on internal port 50000, published on 50010. For `NatPublishedEndpointBrowseTests` (`OPCUA_NAT_ENDPOINT=opc.tcp://10.100.0.35:50010`, #806/#813); see the compose file's header. |
| (this file) | How to run it. |
## Install
@@ -0,0 +1,50 @@
# opc-plc behind a port NAT — the #806/#813 fixture for NatPublishedEndpointBrowseTests
# (tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.OpcUaClient.Browser.IntegrationTests, Fixture=opc-plc-nat).
#
# The point of this stack is what the server ADVERTISES, not what it simulates. opc-plc builds its
# GetEndpoints EndpointUrl from its PLC hostname (--ph; default = the container's short hostname) + its
# listen port, so with
# * a private hostname that cannot resolve from any client (named explicitly via --ph: `.invalid` is
# reserved by RFC 6761 and guaranteed never to resolve, so no resolver search domain or wildcard can make
# the premise pass; the container `hostname:` alone would advertise the bare short name), and
# * a published host port (50010) different from the internal listen port (50000),
# every advertised endpoint is opc.tcp://otopcua-opc-plc-nat-private.invalid:50000/ — unreachable on BOTH host
# and port. A client that follows the advertised URL fails DNS; one that keeps the operator-configured
# host:port (OpcUaEndpointUrlRewriter) reaches the server. The tests assert that premise first.
#
# A SEPARATE stack from ../docker-compose.yml (deployed to /opt/otopcua-opcuaclient-nat, not
# /opt/otopcua-opcuaclient) so bringing it up/down never touches the plain otopcua-opc-plc fixture.
#
# rsync -av tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.OpcUaClient.IntegrationTests/Docker/nat/ \
# dohertj2@10.100.0.35:/opt/otopcua-opcuaclient-nat/
# ssh dohertj2@10.100.0.35 'cd /opt/otopcua-opcuaclient-nat && docker compose up -d'
# OPCUA_NAT_ENDPOINT=opc.tcp://10.100.0.35:50010 dotnet test \
# tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.OpcUaClient.Browser.IntegrationTests --filter Fixture=opc-plc-nat
services:
opc-plc-nat:
image: mcr.microsoft.com/iotedge/opc-plc:2.14.10
container_name: otopcua-opc-plc-nat
hostname: otopcua-opc-plc-nat-private
restart: unless-stopped
labels:
project: lmxopcua
ports:
# published 50010 -> internal 50000: the port half of the NAT. Do NOT make these equal.
- "50010:50000"
command:
- "--pn=50000"
# --ph: the host part of every advertised EndpointUrl (and the server cert's DNS name).
- "--ph=otopcua-opc-plc-nat-private.invalid"
# --ut: also advertise SecurityPolicy=None (the secure policies stay advertised; the secure test uses them).
- "--ut"
# --aa: accept the browser's self-signed client cert. Trust in the OTHER direction (the browser trusting
# this server's cert) is deliberately not automatic — that is what the secure test pins.
- "--aa"
healthcheck:
# The image ships no netstat (the sibling ../docker-compose.yml healthcheck is permanently "unhealthy" for that
# reason); bash's /dev/tcp is there.
test: ["CMD", "bash", "-c", "exec 3<>/dev/tcp/127.0.0.1/50000"]
interval: 5s
timeout: 2s
retries: 10
start_period: 10s
@@ -54,10 +54,28 @@ public sealed class OpcUaClientEndpointRewriteTests
}
[Fact]
public void Host_matching_is_case_insensitive_and_keeps_the_configured_spelling()
public void Host_matching_is_case_insensitive_so_a_case_only_difference_is_not_a_rewrite()
{
// #812: unified with the Client CLI/UI rule — nothing to rewrite, so the advertised string comes back verbatim
// (it used to be re-serialised with the configured spelling; DNS is case-insensitive, so the target is the same).
OpcUaEndpointUrlRewriter.Rewrite("opc.tcp://PLC-A:4840/Server", "OPC.TCP://plc-a:4840/")
.ShouldBe("opc.tcp://plc-a:4840/Server");
.ShouldBe("opc.tcp://PLC-A:4840/Server");
}
[Fact]
public void A_no_op_rewrite_returns_the_advertised_string_verbatim()
{
// #812: no trailing slash appended when nothing changes (the Client CLI/UI's long-standing behaviour).
OpcUaEndpointUrlRewriter.Rewrite("opc.tcp://server:4840", "opc.tcp://server:4840")
.ShouldBe("opc.tcp://server:4840");
}
[Fact]
public void A_rewrite_keeps_the_advertised_path_escaping()
{
// #812: Uri.ToString() un-escapes ("/x%20y" became "/x y"); AbsoluteUri keeps the server's own escaping.
OpcUaEndpointUrlRewriter.Rewrite("opc.tcp://a:4840/x%20y?q=1", "opc.tcp://b:1/")
.ShouldBe("opc.tcp://b:1/x%20y?q=1");
}
[Fact]