docs(infra): add infra/ folder — canonical testing + deployment infra index
Consolidates the scattered infra facts (docker host layout, driver fixtures, the three env-gated live-gate recipes, the Host.IntegrationTests harness, the HistGW local deploy, docker-dev rig, shared GLAuth, deploy) into infra/README.md as the single entry point, cross-linked to dev-environment.md + the deep docs. CLAUDE.md points at it from the Docker Workflow section.
This commit is contained in:
Joseph Doherty
+193
@@ -0,0 +1,193 @@
|
||||
# Infrastructure — testing & deployment
|
||||
|
||||
Canonical index of every piece of infrastructure OtOpcUa's tests and deployments
|
||||
depend on: the shared Docker host, the driver integration fixtures, the env-gated
|
||||
**live-gate** recipes, the integration-test harness, the local dev rig, and the
|
||||
gateway/deploy setups. This file is the single entry point; deeper history lives in
|
||||
the linked docs.
|
||||
|
||||
> **Credentials below are dev-only** (per decision #137) — never production values.
|
||||
> Production uses Integrated Security / gMSA (SQL) and a real directory (LDAP).
|
||||
|
||||
**Related deep docs**
|
||||
- [`docs/v2/dev-environment.md`](../docs/v2/dev-environment.md) — full historical inventory + Windows bootstrap.
|
||||
- [`docs/v2/Galaxy.ParityRig.md`](../docs/v2/Galaxy.ParityRig.md) — mxaccessgw / Galaxy parity-rig runbook.
|
||||
- `docs/drivers/*-Test-Fixture.md` — per-driver fixture coverage maps.
|
||||
- [`../scadaproj/infra/glauth/`](../../scadaproj/infra/glauth/) — the shared GLAuth source-of-truth + deploy runbook.
|
||||
- [`reference_histgw_local_deploy`], [`reference_ww_historian_vm`] (memory) — HistorianGateway deploy detail.
|
||||
|
||||
---
|
||||
|
||||
## 1. Shared Linux Docker host — `10.100.0.35`
|
||||
|
||||
All Docker workloads (central SQL, driver fixtures, integration-harness deps) run on a
|
||||
shared Debian host, **not** on the dev machine. Discover everything with the
|
||||
`project=lmxopcua` label:
|
||||
|
||||
```bash
|
||||
ssh dohertj2@10.100.0.35 'docker ps --filter label=project=lmxopcua --format "{{.Names}}\t{{.Status}}"'
|
||||
```
|
||||
|
||||
- **From this Mac**, drive it over passwordless SSH (`ssh dohertj2@10.100.0.35 'docker …'`).
|
||||
`docker -H ssh://…` does **not** work from the Windows VM (stdio bridge hangs) — the
|
||||
Windows VM uses the `lmxopcua-fix.ps1` helper (`~/bin`) instead; the Mac uses SSH directly.
|
||||
- **Stacks** live at `/opt/otopcua-<driver>/` (root-owned) or `~/otopcua-<driver>/` (user-owned,
|
||||
where `/opt` needs sudo). Current layout:
|
||||
|
||||
| Stack dir | Purpose |
|
||||
|---|---|
|
||||
| `/opt/otopcua-mssql` | central SQL (always-on) |
|
||||
| `/opt/otopcua-modbus` · `/opt/otopcua-abcip` · `/opt/otopcua-s7` · `/opt/otopcua-opcuaclient` | driver fixtures |
|
||||
| `~/otopcua-ablegacy` · `~/otopcua-focas` | driver fixtures (user-owned) |
|
||||
| `~/otopcua-harness` | Host.IntegrationTests real-mode deps (SQL + GLAuth) — see §4 |
|
||||
|
||||
- The repo's `tests/.../Docker/` trees are the **source of truth**; the host dirs are mirrored
|
||||
deployments. Push changes with `rsync`/`lmxopcua-fix sync` before `docker compose up`.
|
||||
|
||||
## 2. Central SQL Server (always-on)
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Container | `otopcua-mssql` (`mcr.microsoft.com/mssql/server:2022-latest`), stack `/opt/otopcua-mssql` |
|
||||
| Endpoint | `10.100.0.35:14330` → `1433` |
|
||||
| Creds (dev) | `sa` / `OtOpcUaDev_2026!` |
|
||||
| Used by | `appsettings.json` `ConfigDb`; DB-backed test suites (`*.Configuration.Tests`, `*.Admin.Tests`, `*.Server.Tests`) |
|
||||
|
||||
Nuclear reset: `ssh dohertj2@10.100.0.35 'cd /opt/otopcua-mssql && docker compose down -v && docker compose up -d'`.
|
||||
|
||||
## 3. Driver integration fixtures
|
||||
|
||||
Each driver's integration suite TCP-probes its fixture at collection init and **skips cleanly**
|
||||
when it isn't running. Bring one up, `dotnet test`, tear down. Repo compose lives under
|
||||
`tests/Drivers/….IntegrationTests/Docker/`.
|
||||
|
||||
| Driver | Fixture image | Endpoint | Bring up (on the host) |
|
||||
|---|---|---|---|
|
||||
| Modbus | `otopcua-pymodbus:3.13.0` | `10.100.0.35:5020` | `docker compose --profile <standard\|dl205\|mitsubishi\|s7_1500> up -d` |
|
||||
| AB CIP | `otopcua-ab-server:libplctag-release` | `10.100.0.35:44818` | `docker compose --profile <controllogix\|compactlogix\|micro800\|guardlogix> up -d` |
|
||||
| AB Legacy (PCCC) | `otopcua-ab-server` (PCCC mode) | `10.100.0.35:44818` | `AB_LEGACY_COMPOSE_PROFILE=<…> docker compose up -d` (stack `~/otopcua-ablegacy`) |
|
||||
| S7 | `otopcua-python-snap7` | `10.100.0.35:1102` | `docker compose --profile s7_1500 up -d` |
|
||||
| OpcUaClient | `mcr.microsoft.com/iotedge/opc-plc:2.14.10` | `opc.tcp://10.100.0.35:50000` | `docker compose up -d` |
|
||||
| FOCAS | `otopcua-focas-sim` (vendored focas-mock) | `10.100.0.35:8193` | `docker compose up -d` (stack `~/otopcua-focas`) |
|
||||
|
||||
**Endpoint overrides** point a suite at a real PLC instead of the sim:
|
||||
`MODBUS_SIM_ENDPOINT` · `AB_SERVER_ENDPOINT` · `S7_SIM_ENDPOINT` · `OPCUA_SIM_ENDPOINT`.
|
||||
|
||||
> **Fixture-cycling gotcha:** profile-gated services can share a port; a plain `docker compose down`
|
||||
> may leave a stale container bound. Force-remove before switching profiles:
|
||||
> `docker rm -f $(docker ps -aq --filter name=otopcua-<driver>)` then `up --force-recreate`.
|
||||
|
||||
## 4. Host.IntegrationTests real-mode harness
|
||||
|
||||
`tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests` runs on EF-InMemory + a stub LDAP by
|
||||
default (zero infra). Real-mode swaps in **SQL Server** + a real **GLAuth** bind. Stack:
|
||||
`~/otopcua-harness` (repo: `tests/Server/.../Host.IntegrationTests/docker-compose.yml` + `glauth/config.toml`).
|
||||
|
||||
| Dep | Container | Endpoint | Notes |
|
||||
|---|---|---|---|
|
||||
| SQL | `otopcua-harness-sql-1` | `10.100.0.35:14331` → 1433 | port 14331 avoids the always-on 14330 |
|
||||
| GLAuth | `otopcua-harness-ldap-1` | `10.100.0.35:3894` → 3893 | GLAuth (swapped in for retired bitnami/openldap, PR #451); users below |
|
||||
|
||||
Harness GLAuth dev users (`glauth/config.toml`, baseDN `dc=zb,dc=local`):
|
||||
`serviceaccount`/`serviceaccount123` (search), `alice`/`alice123` (full-access), `bob`/`bob123` (read-only).
|
||||
|
||||
```bash
|
||||
ssh dohertj2@10.100.0.35 'cd ~/otopcua-harness && docker compose up -d'
|
||||
export OTOPCUA_HARNESS_USE_SQL=1 OTOPCUA_HARNESS_USE_LDAP=1
|
||||
export OTOPCUA_HARNESS_SQL_HOST=10.100.0.35,14331
|
||||
export OTOPCUA_HARNESS_LDAP_HOST=10.100.0.35 # harness uses port 3894
|
||||
dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests
|
||||
ssh dohertj2@10.100.0.35 'cd ~/otopcua-harness && docker compose down'
|
||||
```
|
||||
|
||||
## 5. Live-gate recipes (env-gated destructive tests)
|
||||
|
||||
These prove failure/recovery behavior against **real** infra and skip cleanly offline. Each is
|
||||
gated by env vars and always restores the shared resource in a `finally`.
|
||||
|
||||
### 5a. S7 connect-timeout blackhole (`S7_1500ConnectTimeoutOutageTests`)
|
||||
Proves an S7 subscription survives a SYN-blackhole outage (frozen PLC), which a `docker restart`
|
||||
can't reproduce. `docker pause` gives the ACK-less-established shape — no sudo/VPN.
|
||||
|
||||
```bash
|
||||
ssh dohertj2@10.100.0.35 'cd /opt/otopcua-s7 && docker compose --profile s7_1500 up -d'
|
||||
export S7_TIMEOUT_OUTAGE_START_CMD='ssh dohertj2@10.100.0.35 "docker pause otopcua-python-snap7-s7_1500"'
|
||||
export S7_TIMEOUT_OUTAGE_STOP_CMD='ssh dohertj2@10.100.0.35 "docker unpause otopcua-python-snap7-s7_1500"'
|
||||
dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.S7.IntegrationTests --filter "FullyQualifiedName~ConnectTimeoutOutage"
|
||||
```
|
||||
|
||||
### 5b. LDAP GLAuth outage/recovery (`LdapGlauthOutageLiveTests`)
|
||||
Drives the directory-outage circuit through the production auth path against the harness GLAuth
|
||||
(§4), paused/unpaused. Also verifies the GLAuth swap binds correctly.
|
||||
|
||||
```bash
|
||||
ssh dohertj2@10.100.0.35 'cd ~/otopcua-harness && docker compose up -d ldap'
|
||||
export GLAUTH_LIVE_HOST=10.100.0.35 # port defaults to 3894 (GLAUTH_LIVE_PORT)
|
||||
export GLAUTH_OUTAGE_START_CMD='ssh dohertj2@10.100.0.35 "docker pause otopcua-harness-ldap-1"'
|
||||
export GLAUTH_OUTAGE_STOP_CMD='ssh dohertj2@10.100.0.35 "docker unpause otopcua-harness-ldap-1"'
|
||||
dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests --filter "FullyQualifiedName~LdapGlauthOutageLive"
|
||||
```
|
||||
|
||||
### 5c. HistorianGateway LiveIntegration soak (`Category=LiveIntegration`) — needs VPN
|
||||
Exercises the gateway read/write/alarm/EnsureTags paths against a **real AVEVA historian**. Requires
|
||||
the corporate **VPN up** and a running gateway (see §6). Full deploy detail: [`reference_histgw_local_deploy`].
|
||||
|
||||
```bash
|
||||
export HISTGW_GATEWAY_ENDPOINT=http://localhost:5221 # local gw, h2c
|
||||
export HISTGW_GATEWAY_ALLOW_UNTRUSTED=true # required for h2c
|
||||
export HISTGW_GATEWAY_APIKEY=histgw_placeholder_placeholder # gw ApiKeys:Mode=Disabled
|
||||
export HISTGW_TEST_TAG=SysTimeSec
|
||||
export HISTGW_WRITE_SANDBOX_TAG=HistGW.LiveTest.Sandbox
|
||||
export HISTGW_BOOL_SANDBOX_TAG=HistGW.LiveTest.BoolSandbox
|
||||
export HISTGW_ALARM_SOURCE=OtOpcUaLiveValidation
|
||||
dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests --filter "Category=LiveIntegration"
|
||||
```
|
||||
|
||||
## 6. HistorianGateway — deploy for live testing
|
||||
|
||||
The gateway sidecar runs locally in Docker (`~/Desktop/HistorianGateway/docker/`) pointed at the
|
||||
**real AVEVA historian** over VPN. It's the backend for OtOpcUa's historian read/alarm/continuous paths.
|
||||
|
||||
- **Hosts (VPN up):** historian gRPC CAP `wonder-sql-vd03.zmr.zimmer.com:32565` (TLS); Runtime SQL `:1433`
|
||||
(`wonderapp`/`Wonderapp1`, SQL login); Galaxy GR SQL `wonder-app-vd03.zmr.zimmer.com:1433`.
|
||||
- **Deploy:** publish FDD on host, then `docker compose up -d --build`. gRPC h2c `localhost:5221`,
|
||||
dashboard/health `localhost:5220`, `ApiKeys:Mode=Disabled`. Corporate hostnames go in a gitignored
|
||||
`docker/.env` / secrets, never in committed compose.
|
||||
- **Known deploy fixes** (committed to the gateway repo): FQDN via `.env`; `libgssapi-krb5-2 gss-ntlmssp`
|
||||
in the Dockerfile (historian v6 + SQL paths auth `NAM\dohertj2` via Negotiate/GSSAPI); `RuntimeDb` on.
|
||||
- Full recipe + gotchas: [`reference_histgw_local_deploy`] · historian VM: [`reference_ww_historian_vm`].
|
||||
|
||||
## 7. docker-dev local rig — `docker-dev/docker-compose.yml`
|
||||
|
||||
A 2-node OtOpcUa cluster built from source for AdminUI / OPC UA `/run` verification.
|
||||
|
||||
| Surface | Endpoint | Notes |
|
||||
|---|---|---|
|
||||
| AdminUI | `http://localhost:9200` (via Traefik) | round-robins central-1/2 — **rebuild both** to verify a UI change |
|
||||
| OPC UA | `opc.tcp://localhost:4840` (central-1) · `:4841` (central-2) | data plane |
|
||||
| Login | disabled (`Security__Auth__DisableLogin=true`) | auto-authenticated full-access admin — no sign-in needed |
|
||||
| LDAP (data plane) | shared GLAuth `10.100.0.35:3893` (`DevStubMode=false`) | gates Client.CLI read/write **role** ops only |
|
||||
|
||||
```bash
|
||||
docker compose -f docker-dev/docker-compose.yml up -d --build central-1 central-2 traefik
|
||||
# headless deploy: POST http://localhost:9200/api/deployments (X-Api-Key)
|
||||
docker compose -f docker-dev/docker-compose.yml down -v # reset
|
||||
```
|
||||
|
||||
## 8. Shared GLAuth (data-plane auth) — `10.100.0.35:3893`
|
||||
|
||||
The project-wide LDAP for docker-dev + live OPC UA data-plane auth (distinct from the harness
|
||||
GLAuth on :3894). Container `zb-shared-glauth`, baseDN `dc=zb,dc=local`, bind
|
||||
`cn=serviceaccount,dc=zb,dc=local`/`serviceaccount123`, test users password `password`. **Source of
|
||||
truth + deploy runbook:** [`../scadaproj/infra/glauth/`](../../scadaproj/infra/glauth/).
|
||||
|
||||
## 9. Deployment (production rig)
|
||||
|
||||
The deploy host runs the OtOpcUa server + `mxaccessgw` + the HistorianGateway sidecar as
|
||||
NSSM-wrapped services from `C:\publish\`. Refresh after landing changes with
|
||||
`scripts/install/Refresh-Services.ps1` (stops reverse-dependency order → backs up → publishes →
|
||||
restarts → smoke-verifies ports 4840/4841). Production creds use Integrated Security / gMSA (SQL)
|
||||
and a real directory (LDAP) — never the dev values in this doc.
|
||||
|
||||
> The legacy Wonderware TCP/ArchestrA sidecar was retired; the historian backend is now the
|
||||
> `ZB.MOM.WW.HistorianGateway` sidecar (see §6 and `docs/Historian.md`).
|
||||
Reference in New Issue
Block a user