dohertj2/lmxopcua
1
0
Fork 0
Code Issues 17 Pull Requests Actions 1 Packages Projects Releases Wiki Activity
Files
90bdaa4437b72f26f61f9377f49c9bc8c19ffb44
lmxopcua/infra
T
History
Joseph Doherty cf9ce91e51 test(mtconnect): live-Agent docker fixture + integration suite (Tasks 19+20) 
The first and only thing in the MTConnect workstream that exercises the driver
against a real Agent; everything else runs on canned XML.

Fixture (Docker/): two services, both stock images with this folder bind-mounted.
- `agent`   mtconnect/agent:2.7.0.12 on :5000. NOTE the repo name: the source
            project is mtconnect/cppagent but the PUBLISHED image is
            mtconnect/agent; mtconnect/cppagent does not exist on Docker Hub.
- `adapter` a stdlib SHDR feeder. Required, not optional: the agent image ships
            only the binary + schemas/styles, so a lone Agent answers /probe and
            then reports every observation UNAVAILABLE forever, which can prove
            nothing about reads or streaming.

Devices.xml seeds one DataItem per inference branch (SAMPLE+units, TIME_SERIES,
PART_COUNT/LINE_NUMBER, controlled vocab, DATA_SET, CONDITION), gives every named
item `name != id` (the inverse of the hand-authored unit fixtures), and leaves one
EVENT permanently unfed so UNAVAILABLE -> BadNoCommunication has a live subject.

Suite (12 tests) asserts STRUCTURALLY against the Agent's own /probe response, never
by literal id, so it survives an edit to Devices.xml and can be pointed at a real
machine tool via MTCONNECT_AGENT_ENDPOINT. It skips cleanly (12/12) when no Agent
answers, and each test carries a hard [Fact(Timeout)] so a half-up fixture cannot
wedge a build.

Three findings from bringing the fixture up, each now pinned in a comment:
- agent.cfg must be pure ASCII; one non-ASCII byte in a COMMENT makes the config
  parser reject the whole file with a bare "Failed / Stopped at line: N" and exit.
- `sampleCount` is an attribute of a TIME_SERIES observation, not of a DataItem
  declaration. A real Agent meeting one on a declaration DROPS THE ENTIRE DATA ITEM
  from the device model, so the inference only ever sees null and a live TIME_SERIES
  tag is always variable-length. Asserted, not ignored.
- The Agent's own <Agent> self-model publishes update-rate SAMPLEs that tick whether
  or not any adapter is attached. Excluding them is load-bearing: with them included
  the "live stream delivered a changed value" test passed with the fixture's data
  source deliberately stopped.

MTConnectError-under-HTTP-200 is NOT covered live and cannot be: cppagent 2.7 answers
an unknown device 404 and an out-of-range sequence 400, each with a well-formed error
body. That shape stays canned-only, and the suite says so.
2026-07-27 13:58:54 -04:00
..
README.md
test(mtconnect): live-Agent docker fixture + integration suite (Tasks 19+20)
2026-07-27 13:58:54 -04:00

README.md

Infrastructure — testing & deployment

Canonical index of every piece of infrastructure OtOpcUa's tests and deployments depend on: the shared Docker host, the driver integration fixtures, the env-gated live-gate recipes, the integration-test harness, the local dev rig, and the gateway/deploy setups. This file is the single entry point; deeper history lives in the linked docs.

Credentials below are dev-only (per decision #137) — never production values. Production uses Integrated Security / gMSA (SQL) and a real directory (LDAP).

Related deep docs

  • docs/v2/dev-environment.md — full historical inventory + Windows bootstrap.
  • docs/v2/Galaxy.ParityRig.md — mxaccessgw / Galaxy parity-rig runbook.
  • docs/drivers/*-Test-Fixture.md — per-driver fixture coverage maps.
  • ../scadaproj/infra/glauth/ — the shared GLAuth source-of-truth + deploy runbook.
  • reference_histgw_local_deploy, reference_ww_historian_vm (memory) — HistorianGateway deploy detail.

1. Shared Linux Docker host — 10.100.0.35

All Docker workloads (central SQL, driver fixtures, integration-harness deps) run on a shared Debian host, not on the dev machine. Discover everything with the project=lmxopcua label:

ssh dohertj2@10.100.0.35 'docker ps --filter label=project=lmxopcua --format "{{.Names}}\t{{.Status}}"'

  • From this Mac, drive it over passwordless SSH (ssh dohertj2@10.100.0.35 'docker …'). docker -H ssh://… does not work from the Windows VM (stdio bridge hangs) — the Windows VM uses the lmxopcua-fix.ps1 helper (~/bin) instead; the Mac uses SSH directly.

  • Stacks live at /opt/otopcua-<driver>/ (root-owned) or ~/otopcua-<driver>/ (user-owned, where /opt needs sudo). Current layout:

    Stack dir Purpose
    /opt/otopcua-mssql central SQL (always-on)
    /opt/otopcua-modbus · /opt/otopcua-abcip · /opt/otopcua-s7 · /opt/otopcua-opcuaclient · /opt/otopcua-mqtt · /opt/otopcua-mtconnect driver fixtures
    ~/otopcua-ablegacy · ~/otopcua-focas driver fixtures (user-owned)
    ~/otopcua-harness Host.IntegrationTests real-mode deps (SQL + GLAuth) — see §4

  • The repo's tests/.../Docker/ trees are the source of truth; the host dirs are mirrored deployments. Push changes with rsync/lmxopcua-fix sync before docker compose up.

2. Central SQL Server (always-on)
Container otopcua-mssql (mcr.microsoft.com/mssql/server:2022-latest), stack /opt/otopcua-mssql
Endpoint 10.100.0.35:14330 → 1433
Creds (dev) sa / OtOpcUaDev_2026!
Used by appsettings.json ConfigDb; DB-backed test suites (*.Configuration.Tests, *.Admin.Tests, *.Server.Tests)

Nuclear reset: ssh dohertj2@10.100.0.35 'cd /opt/otopcua-mssql && docker compose down -v && docker compose up -d'.

3. Driver integration fixtures

Each driver's integration suite TCP-probes its fixture at collection init and skips cleanly when it isn't running. Bring one up, dotnet test, tear down. Repo compose lives under tests/Drivers/…​.IntegrationTests/Docker/.

Driver Fixture image Endpoint Bring up (on the host)
Modbus otopcua-pymodbus:3.13.0 10.100.0.35:5020 docker compose --profile <standard|dl205|mitsubishi|s7_1500> up -d
AB CIP otopcua-ab-server:libplctag-release 10.100.0.35:44818 docker compose --profile <controllogix|compactlogix|micro800|guardlogix> up -d
AB Legacy (PCCC) otopcua-ab-server (PCCC mode) 10.100.0.35:44818 AB_LEGACY_COMPOSE_PROFILE=<…> docker compose up -d (stack ~/otopcua-ablegacy)
S7 otopcua-python-snap7 10.100.0.35:1102 docker compose --profile s7_1500 up -d
OpcUaClient mcr.microsoft.com/iotedge/opc-plc:2.14.10 opc.tcp://10.100.0.35:50000 docker compose up -d
FOCAS otopcua-focas-sim (vendored focas-mock) 10.100.0.35:8193 docker compose up -d (stack ~/otopcua-focas)
MQTT eclipse-mosquitto:2.0.22 (+ a publisher sidecar) 10.100.0.35:8883 TLS+auth · :1883 plaintext+auth MQTT_FIXTURE_USERNAME=otopcua MQTT_FIXTURE_PASSWORD=<pw> docker compose up -d (stack /opt/otopcua-mqtt)
MQTT — Sparkplug B otopcua-sparkplug-sim (project-owned C# edge-node simulator) same broker; group OtOpcUaSim, edge nodes EdgeA / EdgeB, EdgeA device Filler1, ~2 s cadence docker compose --profile sparkplug up -d (same stack). Answers rebirth NCMDs; restart it to force a fresh birth
MTConnect mtconnect/agent:2.7.0.12 + python:3.13-alpine adapter http://10.100.0.35:5000 docker compose up -d --wait

Endpoint overrides point a suite at a real PLC instead of the sim: MODBUS_SIM_ENDPOINT · AB_SERVER_ENDPOINT · S7_SIM_ENDPOINT · OPCUA_SIM_ENDPOINT · MQTT_FIXTURE_ENDPOINT / MQTT_FIXTURE_PLAIN_ENDPOINT · MTCONNECT_AGENT_ENDPOINT.

MQTT needs credentials + material, unlike the other fixtures. The broker has no anonymous fallback on either listener (deliberate — an anonymous broker would let a bad auth config pass), so it will not even start until ./gen-fixture-material.sh has written secrets/ (password file + CA + server cert/key, gitignored) and MQTT_FIXTURE_USERNAME/MQTT_FIXTURE_PASSWORD are exported at docker compose up. The live suite additionally wants MQTT_FIXTURE_CA_CERT pointing at a local copy of the CA: scp dohertj2@10.100.0.35:/opt/otopcua-mqtt/secrets/ca.crt /tmp/mqtt-fixture-ca.crt. It is also the only fixture whose services carry the project: lmxopcua label (see CLAUDE.md).

MTConnect is a TWO-service stack. The mtconnect/agent image ships no simulator, so a lone Agent answers /probe and reports every observation UNAVAILABLE forever. The adapter service (a stdlib SHDR feeder, Docker/adapter.py) is the data source; the Agent dials out to it. Its suite also probes over HTTP, not TCP — an Agent's port accepts connections before its device model is parsed. Full detail + the seeded device model: tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/Docker/README.md.

Fixture-cycling gotcha: profile-gated services can share a port; a plain docker compose down may leave a stale container bound. Force-remove before switching profiles: docker rm -f $(docker ps -aq --filter name=otopcua-<driver>) then up --force-recreate.

4. Host.IntegrationTests real-mode harness

tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests runs on EF-InMemory + a stub LDAP by default (zero infra). Real-mode swaps in SQL Server + a real GLAuth bind. Stack: ~/otopcua-harness (repo: tests/Server/.../Host.IntegrationTests/docker-compose.yml + glauth/config.toml).

Dep Container Endpoint Notes
SQL otopcua-harness-sql-1 10.100.0.35:14331 → 1433 port 14331 avoids the always-on 14330
GLAuth otopcua-harness-ldap-1 10.100.0.35:3894 → 3893 GLAuth (swapped in for retired bitnami/openldap, PR #451); users below

Harness GLAuth dev users (glauth/config.toml, baseDN dc=zb,dc=local): serviceaccount/serviceaccount123 (search), alice/alice123 (full-access), bob/bob123 (read-only).

ssh dohertj2@10.100.0.35 'cd ~/otopcua-harness && docker compose up -d'
export OTOPCUA_HARNESS_USE_SQL=1  OTOPCUA_HARNESS_USE_LDAP=1
export OTOPCUA_HARNESS_SQL_HOST=10.100.0.35,14331
export OTOPCUA_HARNESS_LDAP_HOST=10.100.0.35        # harness uses port 3894
dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests
ssh dohertj2@10.100.0.35 'cd ~/otopcua-harness && docker compose down'

5. Live-gate recipes (env-gated destructive tests)

These prove failure/recovery behavior against real infra and skip cleanly offline. Each is gated by env vars and always restores the shared resource in a finally.

5a. S7 connect-timeout blackhole (S7_1500ConnectTimeoutOutageTests)

Proves an S7 subscription survives a SYN-blackhole outage (frozen PLC), which a docker restart can't reproduce. docker pause gives the ACK-less-established shape — no sudo/VPN.

ssh dohertj2@10.100.0.35 'cd /opt/otopcua-s7 && docker compose --profile s7_1500 up -d'
export S7_TIMEOUT_OUTAGE_START_CMD='ssh dohertj2@10.100.0.35 "docker pause otopcua-python-snap7-s7_1500"'
export S7_TIMEOUT_OUTAGE_STOP_CMD='ssh dohertj2@10.100.0.35 "docker unpause otopcua-python-snap7-s7_1500"'
dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.S7.IntegrationTests --filter "FullyQualifiedName~ConnectTimeoutOutage"

5b. LDAP GLAuth outage/recovery (LdapGlauthOutageLiveTests)

Drives the directory-outage circuit through the production auth path against the harness GLAuth (§4), paused/unpaused. Also verifies the GLAuth swap binds correctly.

ssh dohertj2@10.100.0.35 'cd ~/otopcua-harness && docker compose up -d ldap'
export GLAUTH_LIVE_HOST=10.100.0.35   # port defaults to 3894 (GLAUTH_LIVE_PORT)
export GLAUTH_OUTAGE_START_CMD='ssh dohertj2@10.100.0.35 "docker pause otopcua-harness-ldap-1"'
export GLAUTH_OUTAGE_STOP_CMD='ssh dohertj2@10.100.0.35 "docker unpause otopcua-harness-ldap-1"'
dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests --filter "FullyQualifiedName~LdapGlauthOutageLive"

5c. HistorianGateway LiveIntegration soak (Category=LiveIntegration) — needs VPN

Exercises the gateway read/write/alarm/EnsureTags paths against a real AVEVA historian. Requires the corporate VPN up and a running gateway (see §6). Full deploy detail: reference_histgw_local_deploy.

export HISTGW_GATEWAY_ENDPOINT=http://localhost:5221   # local gw, h2c
export HISTGW_GATEWAY_ALLOW_UNTRUSTED=true              # required for h2c
export HISTGW_GATEWAY_APIKEY=histgw_placeholder_placeholder   # gw ApiKeys:Mode=Disabled
export HISTGW_TEST_TAG=SysTimeSec
export HISTGW_WRITE_SANDBOX_TAG=HistGW.LiveTest.Sandbox
export HISTGW_BOOL_SANDBOX_TAG=HistGW.LiveTest.BoolSandbox
export HISTGW_ALARM_SOURCE=OtOpcUaLiveValidation
dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests --filter "Category=LiveIntegration"

6. HistorianGateway — deploy for live testing

The gateway sidecar runs locally in Docker (~/Desktop/HistorianGateway/docker/) pointed at the real AVEVA historian over VPN. It's the backend for OtOpcUa's historian read/alarm/continuous paths.

  • Hosts (VPN up): historian gRPC CAP wonder-sql-vd03.zmr.zimmer.com:32565 (TLS); Runtime SQL :1433 (wonderapp/Wonderapp1, SQL login); Galaxy GR SQL wonder-app-vd03.zmr.zimmer.com:1433.
  • Deploy: publish FDD on host, then docker compose up -d --build. gRPC h2c localhost:5221, dashboard/health localhost:5220, ApiKeys:Mode=Disabled. Corporate hostnames go in a gitignored docker/.env / secrets, never in committed compose.
  • Known deploy fixes (committed to the gateway repo): FQDN via .env; libgssapi-krb5-2 gss-ntlmssp in the Dockerfile (historian v6 + SQL paths auth NAM\dohertj2 via Negotiate/GSSAPI); RuntimeDb on.
  • Full recipe + gotchas: reference_histgw_local_deploy · historian VM: reference_ww_historian_vm.

7. docker-dev local rig — docker-dev/docker-compose.yml

A 2-node OtOpcUa cluster built from source for AdminUI / OPC UA /run verification.

Surface Endpoint Notes
AdminUI http://localhost:9200 (via Traefik) round-robins central-1/2 — rebuild both to verify a UI change
OPC UA opc.tcp://localhost:4840 (central-1) · :4841 (central-2) data plane
Login disabled (Security__Auth__DisableLogin=true) auto-authenticated full-access admin — no sign-in needed
LDAP (data plane) shared GLAuth 10.100.0.35:3893 (DevStubMode=false) gates Client.CLI read/write role ops only 
docker compose -f docker-dev/docker-compose.yml up -d --build central-1 central-2 traefik
# headless deploy: POST http://localhost:9200/api/deployments  (X-Api-Key)
docker compose -f docker-dev/docker-compose.yml down -v   # reset

8. Shared GLAuth (data-plane auth) — 10.100.0.35:3893

The project-wide LDAP for docker-dev + live OPC UA data-plane auth (distinct from the harness GLAuth on :3894). Container zb-shared-glauth, baseDN dc=zb,dc=local, bind cn=serviceaccount,dc=zb,dc=local/serviceaccount123, test users password password. Source of truth + deploy runbook: ../scadaproj/infra/glauth/.

9. Deployment (production rig)

The deploy host runs the OtOpcUa server + mxaccessgw + the HistorianGateway sidecar as NSSM-wrapped services from C:\publish\. Refresh after landing changes with scripts/install/Refresh-Services.ps1 (stops reverse-dependency order → backs up → publishes → restarts → smoke-verifies ports 4840/4841). Production creds use Integrated Security / gMSA (SQL) and a real directory (LDAP) — never the dev values in this doc.

The legacy Wonderware TCP/ArchestrA sidecar was retired; the historian backend is now the ZB.MOM.WW.HistorianGateway sidecar (see §6 and docs/Historian.md).

Reference in New Issue View Git Blame Copy Permalink