lmxopcua/infra

Infrastructure — testing & deployment

Canonical index of every piece of infrastructure OtOpcUa's tests and deployments depend on: the shared Docker host, the driver integration fixtures, the env-gated live-gate recipes, the integration-test harness, the local dev rig, and the gateway/deploy setups. This file is the single entry point; deeper history lives in the linked docs.

Credentials below are dev-only (per decision #137) — never production values. Production uses Integrated Security / gMSA (SQL) and a real directory (LDAP).

Related deep docs

• docs/v2/dev-environment.md — full historical inventory + Windows bootstrap.
• docs/v2/Galaxy.ParityRig.md — mxaccessgw / Galaxy parity-rig runbook.
• docs/drivers/*-Test-Fixture.md — per-driver fixture coverage maps.
• ../scadaproj/infra/glauth/ — the shared GLAuth source-of-truth + deploy runbook.
• [reference_histgw_local_deploy], [reference_ww_historian_vm] (memory) — HistorianGateway deploy detail.

---

1. Shared Linux Docker host — 10.100.0.35

All Docker workloads (central SQL, driver fixtures, integration-harness deps) run on a shared Debian host, not on the dev machine. Discover everything with the project=lmxopcua label:

ssh dohertj2@10.100.0.35 'docker ps --filter label=project=lmxopcua --format "{{.Names}}\t{{.Status}}"'

• From this Mac, drive it over passwordless SSH (ssh dohertj2@10.100.0.35 'docker …'). docker -H ssh://… does not work from the Windows VM (stdio bridge hangs) — the Windows VM uses the lmxopcua-fix.ps1 helper (~/bin) instead; the Mac uses SSH directly.

• Stacks live at /opt/otopcua-<driver>/ (root-owned) or ~/otopcua-<driver>/ (user-owned, where /opt needs sudo). Current layout:

  Stack dir	Purpose
  /opt/otopcua-mssql	central SQL (always-on)
  /opt/otopcua-modbus · /opt/otopcua-abcip · /opt/otopcua-s7 · /opt/otopcua-opcuaclient	driver fixtures
  ~/otopcua-ablegacy · ~/otopcua-focas	driver fixtures (user-owned)
  ~/otopcua-harness	Host.IntegrationTests real-mode deps (SQL + GLAuth) — see §4

• The repo's tests/.../Docker/ trees are the source of truth; the host dirs are mirrored deployments. Push changes with rsync/lmxopcua-fix sync before docker compose up.

2. Central SQL Server (always-on)

Container	otopcua-mssql (mcr.microsoft.com/mssql/server:2022-latest), stack /opt/otopcua-mssql
Endpoint	10.100.0.35:14330 → 1433
Creds (dev)	sa / OtOpcUaDev_2026!
Used by	appsettings.json ConfigDb; DB-backed test suites (*.Configuration.Tests, *.Admin.Tests, *.Server.Tests)

Nuclear reset: ssh dohertj2@10.100.0.35 'cd /opt/otopcua-mssql && docker compose down -v && docker compose up -d'.

3. Driver integration fixtures

Each driver's integration suite TCP-probes its fixture at collection init and skips cleanly when it isn't running. Bring one up, dotnet test, tear down. Repo compose lives under tests/Drivers/…​.IntegrationTests/Docker/.

Driver	Fixture image	Endpoint	Bring up (on the host)
Modbus	otopcua-pymodbus:3.13.0	10.100.0.35:5020	docker compose --profile <standard|dl205|mitsubishi|s7_1500> up -d
AB CIP	otopcua-ab-server:libplctag-release	10.100.0.35:44818	docker compose --profile <controllogix|compactlogix|micro800|guardlogix> up -d
AB Legacy (PCCC)	otopcua-ab-server (PCCC mode)	10.100.0.35:44818	AB_LEGACY_COMPOSE_PROFILE=<…> docker compose up -d (stack ~/otopcua-ablegacy)
S7	otopcua-python-snap7	10.100.0.35:1102	docker compose --profile s7_1500 up -d
OpcUaClient	mcr.microsoft.com/iotedge/opc-plc:2.14.10	opc.tcp://10.100.0.35:50000	docker compose up -d
FOCAS	otopcua-focas-sim (vendored focas-mock)	10.100.0.35:8193	docker compose up -d (stack ~/otopcua-focas)

Endpoint overrides point a suite at a real PLC instead of the sim: MODBUS_SIM_ENDPOINT · AB_SERVER_ENDPOINT · S7_SIM_ENDPOINT · OPCUA_SIM_ENDPOINT.

Fixture-cycling gotcha: profile-gated services can share a port; a plain docker compose down may leave a stale container bound. Force-remove before switching profiles: docker rm -f $(docker ps -aq --filter name=otopcua-<driver>) then up --force-recreate.

4. Host.IntegrationTests real-mode harness

tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests runs on EF-InMemory + a stub LDAP by default (zero infra). Real-mode swaps in SQL Server + a real GLAuth bind. Stack: ~/otopcua-harness (repo: tests/Server/.../Host.IntegrationTests/docker-compose.yml + glauth/config.toml).

Dep	Container	Endpoint	Notes
SQL	otopcua-harness-sql-1	10.100.0.35:14331 → 1433	port 14331 avoids the always-on 14330
GLAuth	otopcua-harness-ldap-1	10.100.0.35:3894 → 3893	GLAuth (swapped in for retired bitnami/openldap, PR #451); users below

Harness GLAuth dev users (glauth/config.toml, baseDN dc=zb,dc=local): serviceaccount/serviceaccount123 (search), alice/alice123 (full-access), bob/bob123 (read-only).

ssh dohertj2@10.100.0.35 'cd ~/otopcua-harness && docker compose up -d'
export OTOPCUA_HARNESS_USE_SQL=1  OTOPCUA_HARNESS_USE_LDAP=1
export OTOPCUA_HARNESS_SQL_HOST=10.100.0.35,14331
export OTOPCUA_HARNESS_LDAP_HOST=10.100.0.35        # harness uses port 3894
dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests
ssh dohertj2@10.100.0.35 'cd ~/otopcua-harness && docker compose down'

5. Live-gate recipes (env-gated destructive tests)

These prove failure/recovery behavior against real infra and skip cleanly offline. Each is gated by env vars and always restores the shared resource in a finally.

5a. S7 connect-timeout blackhole (S7_1500ConnectTimeoutOutageTests)

Proves an S7 subscription survives a SYN-blackhole outage (frozen PLC), which a docker restart can't reproduce. docker pause gives the ACK-less-established shape — no sudo/VPN.

ssh dohertj2@10.100.0.35 'cd /opt/otopcua-s7 && docker compose --profile s7_1500 up -d'
export S7_TIMEOUT_OUTAGE_START_CMD='ssh dohertj2@10.100.0.35 "docker pause otopcua-python-snap7-s7_1500"'
export S7_TIMEOUT_OUTAGE_STOP_CMD='ssh dohertj2@10.100.0.35 "docker unpause otopcua-python-snap7-s7_1500"'
dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.S7.IntegrationTests --filter "FullyQualifiedName~ConnectTimeoutOutage"

5b. LDAP GLAuth outage/recovery (LdapGlauthOutageLiveTests)

Drives the directory-outage circuit through the production auth path against the harness GLAuth (§4), paused/unpaused. Also verifies the GLAuth swap binds correctly.

ssh dohertj2@10.100.0.35 'cd ~/otopcua-harness && docker compose up -d ldap'
export GLAUTH_LIVE_HOST=10.100.0.35   # port defaults to 3894 (GLAUTH_LIVE_PORT)
export GLAUTH_OUTAGE_START_CMD='ssh dohertj2@10.100.0.35 "docker pause otopcua-harness-ldap-1"'
export GLAUTH_OUTAGE_STOP_CMD='ssh dohertj2@10.100.0.35 "docker unpause otopcua-harness-ldap-1"'
dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests --filter "FullyQualifiedName~LdapGlauthOutageLive"

5c. HistorianGateway LiveIntegration soak (Category=LiveIntegration) — needs VPN

Exercises the gateway read/write/alarm/EnsureTags paths against a real AVEVA historian. Requires the corporate VPN up and a running gateway (see §6). Full deploy detail: [reference_histgw_local_deploy].

export HISTGW_GATEWAY_ENDPOINT=http://localhost:5221   # local gw, h2c
export HISTGW_GATEWAY_ALLOW_UNTRUSTED=true              # required for h2c
export HISTGW_GATEWAY_APIKEY=histgw_placeholder_placeholder   # gw ApiKeys:Mode=Disabled
export HISTGW_TEST_TAG=SysTimeSec
export HISTGW_WRITE_SANDBOX_TAG=HistGW.LiveTest.Sandbox
export HISTGW_BOOL_SANDBOX_TAG=HistGW.LiveTest.BoolSandbox
export HISTGW_ALARM_SOURCE=OtOpcUaLiveValidation
dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests --filter "Category=LiveIntegration"

6. HistorianGateway — deploy for live testing

The gateway sidecar runs locally in Docker (~/Desktop/HistorianGateway/docker/) pointed at the real AVEVA historian over VPN. It's the backend for OtOpcUa's historian read/alarm/continuous paths.

• Hosts (VPN up): historian gRPC CAP wonder-sql-vd03.zmr.zimmer.com:32565 (TLS); Runtime SQL :1433 (wonderapp/Wonderapp1, SQL login); Galaxy GR SQL wonder-app-vd03.zmr.zimmer.com:1433.
• Deploy: publish FDD on host, then docker compose up -d --build. gRPC h2c localhost:5221, dashboard/health localhost:5220, ApiKeys:Mode=Disabled. Corporate hostnames go in a gitignored docker/.env / secrets, never in committed compose.
• Known deploy fixes (committed to the gateway repo): FQDN via .env; libgssapi-krb5-2 gss-ntlmssp in the Dockerfile (historian v6 + SQL paths auth NAM\dohertj2 via Negotiate/GSSAPI); RuntimeDb on.
• Full recipe + gotchas: [reference_histgw_local_deploy] · historian VM: [reference_ww_historian_vm].

7. docker-dev local rig — docker-dev/docker-compose.yml

A 2-node OtOpcUa cluster built from source for AdminUI / OPC UA /run verification.

Surface	Endpoint	Notes
AdminUI	http://localhost:9200 (via Traefik)	round-robins central-1/2 — rebuild both to verify a UI change
OPC UA	opc.tcp://localhost:4840 (central-1) · :4841 (central-2)	data plane
Login	disabled (Security__Auth__DisableLogin=true)	auto-authenticated full-access admin — no sign-in needed
LDAP (data plane)	shared GLAuth 10.100.0.35:3893 (DevStubMode=false)	gates Client.CLI read/write role ops only

docker compose -f docker-dev/docker-compose.yml up -d --build central-1 central-2 traefik
# headless deploy: POST http://localhost:9200/api/deployments  (X-Api-Key)
docker compose -f docker-dev/docker-compose.yml down -v   # reset

8. Shared GLAuth (data-plane auth) — 10.100.0.35:3893

The project-wide LDAP for docker-dev + live OPC UA data-plane auth (distinct from the harness GLAuth on :3894). Container zb-shared-glauth, baseDN dc=zb,dc=local, bind cn=serviceaccount,dc=zb,dc=local/serviceaccount123, test users password password. Source of truth + deploy runbook: ../scadaproj/infra/glauth/.

9. Deployment (production rig)

The deploy host runs the OtOpcUa server + mxaccessgw + the HistorianGateway sidecar as NSSM-wrapped services from C:\publish\. Refresh after landing changes with scripts/install/Refresh-Services.ps1 (stops reverse-dependency order → backs up → publishes → restarts → smoke-verifies ports 4840/4841). Production creds use Integrated Security / gMSA (SQL) and a real directory (LDAP) — never the dev values in this doc.

The legacy Wonderware TCP/ArchestrA sidecar was retired; the historian backend is now the ZB.MOM.WW.HistorianGateway sidecar (see §6 and docs/Historian.md).