Files
ScadaBridge/docker-env2
Joseph Doherty c3c1f6e870 chore(docker): windev MxGateway API key no longer in the seed scripts
docker/seed-sites.sh and docker-env2/seed-sites.sh hard-coded the windev
gateway API key. They now read MXGW_APIKEY from the environment, or from
a private file (default ~/.config/scadabridge/windev-mxgw-apikey,
override MXGW_APIKEY_FILE). With no key the MxGateway connection step is
skipped and the rest of the seed runs. READMEs note where to put it.

The key remains in git history (2 commits), so it should be rotated on
the windev gateway.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 00:00:03 -05:00
..

ScadaBridge Env2 Docker Infrastructure

A second Docker deployment of a minimal ScadaBridge cluster topology, designed to run concurrently with the primary docker/ stack so the Transport (#24) feature can be exercised end-to-end across two real environments.

See docs/plans/2026-05-24-second-environment-design.md for the design rationale.

Cluster Topology

              ┌───────────────────┐
              │  Traefik LB :9100 │  ◄── CLI / Browser
              │  Dashboard :8181  │
              └────────┬──────────┘
                       │ routes to active node
┌──────────────────────┼──────────────────────────────┐
│            Env2 Central Cluster                     │
│                                                     │
│  ┌─────────────────┐     ┌─────────────────┐        │
│  │ env2-central-a   │◄──►│ env2-central-b   │       │
│  │  Web UI :9101    │     │  Web UI :9102    │       │
│  │  Akka   :9111    │     │  Akka   :9112    │       │
│  └────────┬─────────┘     └─────────────────┘       │
│           │                                         │
└───────────┼─────────────────────────────────────────┘
            │ Akka.NET Remoting
            ▼
┌────────────────────┐
│  Env2 Site-X       │
│  (Env2 Site X)     │
│                    │
│  node-a ◄──► node-b│
│  Akka :9121 :9122  │
│  gRPC :9123 :9124  │
└────────────────────┘

Port Allocation

Env2 host ports are the primary's ports + 100. Both stacks can run simultaneously.

Node Container Name Host Web Host Akka Host gRPC Internal
Traefik LB scadabridge-env2-traefik 9100 — — 80 (proxy), 8080 (dashboard host:8181)
Central A scadabridge-env2-central-a 9101 9111 9113 5000 (web), 8081 (Akka), 8083 (gRPC)
Central B scadabridge-env2-central-b 9102 9112 9114 5000 (web), 8081 (Akka), 8083 (gRPC)
Site-X A scadabridge-env2-site-x-a — 9121 9123 8082 (Akka), 8083 (gRPC)
Site-X B scadabridge-env2-site-x-b — 9122 9124 8082 (Akka), 8083 (gRPC)

Shared Infrastructure

Env2 attaches to the existing scadabridge-net Docker bridge network and reuses these primary infra containers:

Service Container What env2 uses it for
MS SQL scadabridge-mssql Env2-specific databases ScadaBridgeConfig2 / ScadaBridgeMachineData2
SMTP scadabridge-smtp Notification capture in Mailpit (env2 emails distinguishable by FromAddress)
OPC UA scadabridge-opcua Simulated tags for site-x data connections
REST API scadabridge-restapi External REST API testing

LDAP is not one of the reused infra/ containers (that stack's ldap service is retired/commented out). Env2's central nodes bind to the same local redundant GLAuth pair as docker/ — zb-ldap-primary on host.docker.internal:3893 with zb-ldap-backup on host.docker.internal:3894 as FallbackServers (source of truth: scadaproj/infra/glauth-redundant); same test users, same bind credentials as the primary stack.

Commands

First-Time Setup

# 1. Make sure primary infra is up (creates scadabridge-net, scadabridge-mssql, etc.)
cd infra && docker compose up -d && cd ..

# 2. Build image + create env2 databases + deploy env2 containers
bash docker-env2/deploy.sh

# 3. Seed env2's single test site (first-time only)
bash docker-env2/seed-sites.sh

# 4. Optional: live data + store-and-forward load for site-x (opc-plc, a read-only
#    Galaxy attribute, and the scadabridge-env2-sfsink HTTP sink) — the #116 soak load
bash docker-env2/seed-data-path.sh

The MxGateway data connections need the windev gateway API key, which is not in the repo. Put it in ~/.config/scadabridge/windev-mxgw-apikey (mode 600), point MXGW_APIKEY_FILE at another file, or export MXGW_APIKEY. Without a key the script skips the MxGateway connection step and seeds everything else.

Site-x's LocalDb is replicated (node-a dials node-b on 8083, same dev key on both), like the rig's site-a. Without it a site failover brings up an active node with no deployed instances, connections or S&F buffer.

The seed also creates a native alarm source demo: a MxAlarmDemo template with a GalaxyAlarms native alarm source bound to the shared MxGateway connection, plus a deployed MxAlarmDemo-1 instance on site-x. This exercises the read-only native alarm mirror end-to-end — mirrored alarms appear live in the Central UI Debug View with severity and condition badges. Verify the binding with:

scadabridge --url http://localhost:9100 template native-alarm-source list --template-id <id>

After Code Changes

bash docker-env2/deploy.sh

The Docker build is shared with the primary stack — if you've just run docker/deploy.sh, the env2 build hits a fully cached image.

Running Both Stacks Concurrently

bash docker/deploy.sh         # primary
bash docker-env2/deploy.sh    # env2

Both UIs are now reachable:

Own image tag, and the leader-auto-down soak (ScadaBridge#116)

SCADABRIDGE_IMAGE=<repo:tag> makes docker/build.sh tag that image and this compose file run it, so an env2 build never retags the scadabridge:latest the primary rig runs. Unset keeps the old shared-image behaviour.

Since 2026-11-12 env2 is the soak environment for the leader-auto-down rollout and runs scadabridge:env2-116 with the opt-in overlay docker-compose.leader-auto-down.yml. Keep both when recreating nodes, and switch both pairs together:

SCADABRIDGE_IMAGE=scadabridge:env2-116 docker compose \
  -f docker-env2/docker-compose.yml -f docker-env2/docker-compose.leader-auto-down.yml \
  up -d --force-recreate --no-deps central-a central-b site-x-a site-x-b

Plain deploy.sh goes back to scadabridge:latest on the default auto-down. Drill: DRILL_ENV=env2 DRILL_PAIR=central|site-x bash docker/failover-drill.sh. Record: docs/plans/2026-11-12-leader-auto-down-env2-drill.md.

Teardown

bash docker-env2/teardown.sh

Containers stop; node data lives in named Docker volumes (scadabridge-env2-*-data) and logs in node directories, both left in place. To also remove the persistent data:

# Destructive: deletes all env2 SQLite data (site config cache, S&F buffers, LocalDb).
docker volume rm $(docker volume ls -q --filter name=scadabridge-env2-)
rm -rf docker-env2/*/logs

To also drop the env2 databases:

docker exec scadabridge-mssql /opt/mssql-tools18/bin/sqlcmd \
    -S localhost -U sa -P 'ScadaBridge_Dev1#' -C \
    -Q "DROP DATABASE ScadaBridgeConfig2; DROP DATABASE ScadaBridgeMachineData2;"

CLI Access

dotnet run --project src/ZB.MOM.WW.ScadaBridge.CLI -- \
    --url http://localhost:9100 \
    --username multi-role --password password \
    template list

View Logs

docker compose -f docker-env2/docker-compose.yml logs -f
docker logs -f scadabridge-env2-central-a

Test Users

Same as primary (env2 shares the local redundant GLAuth pair, scadaproj/infra/glauth-redundant). See "Shared Infrastructure" above and primary docker/README.md.

Transport Testing Workflow

See docs/plans/2026-05-24-second-environment-verification.md for the manual golden-path checklist.

What's Different from Primary

  • Single site (site-x) instead of three (site-a/b/c).
  • Host port range 91XX vs primary 90XX.
  • Container names prefixed scadabridge-env2-.
  • Databases ScadaBridgeConfig2 / ScadaBridgeMachineData2 on the shared scadabridge-mssql.
  • Transport.SourceEnvironment = "docker-cluster-env2" (stamped into exported bundle manifests).
  • Distinct Security.JwtSigningKey (sessions cannot cross envs).