Files
ScadaBridge/docker-env2/README.md
T
Joseph Doherty efade2eab0 feat(cluster)!: leader-auto-down is the shipped default downing strategy (#116)
SplitBrainResolverStrategy now defaults to leader-auto-down (the in-repo
LeaderAutoDowningProvider, #106, with the #132 shut-out restart on). auto-down and
keep-oldest stay supported. Drilled on the docker rig, docker-env2 and the QA pair
(central + site, kill and stop-and-hold) before the flip.

- ClusterOptions default + validator/HOCON comments updated.
- Shipped Host appsettings.Central/Site.json and every docker/ and docker-env2/
  node appsettings now say leader-auto-down. The leader-auto-down overlays are
  kept as no-ops (drill records reference them); new docker-compose.auto-down.yml
  overlays keep Akka AutoDowning available for comparison drills.
- Tests: default asserts leader-auto-down; new HOCON test for an unset strategy;
  TwoNodeClusterFixture defaults to the production posture.
- Docs: CLAUDE.md, Component-ClusterInfrastructure, components/Host + Cluster,
  README, failover procedures, Traefik, docker + env2 READMEs; register row 70 closed.

No deploy needed: DEV, QA and both rigs set the key explicitly.

Claude-Session: https://claude.ai/code/session_01GT67AFKC45MBz1wPZnVbX8
2026-11-15 19:57:45 -05:00

9.0 KiB

ScadaBridge Env2 Docker Infrastructure

A second Docker deployment of a minimal ScadaBridge cluster topology, designed to run concurrently with the primary docker/ stack so the Transport (#24) feature can be exercised end-to-end across two real environments.

See docs/plans/2026-05-24-second-environment-design.md for the design rationale.

Cluster Topology

              ┌───────────────────┐
              │  Traefik LB :9100 │  ◄── CLI / Browser
              │  Dashboard :8181  │
              └────────┬──────────┘
                       │ routes to active node
┌──────────────────────┼──────────────────────────────┐
│            Env2 Central Cluster                     │
│                                                     │
│  ┌─────────────────┐     ┌─────────────────┐        │
│  │ env2-central-a   │◄──►│ env2-central-b   │       │
│  │  Web UI :9101    │     │  Web UI :9102    │       │
│  │  Akka   :9111    │     │  Akka   :9112    │       │
│  └────────┬─────────┘     └─────────────────┘       │
│           │                                         │
└───────────┼─────────────────────────────────────────┘
            │ Akka.NET Remoting
            ▼
┌────────────────────┐
│  Env2 Site-X       │
│  (Env2 Site X)     │
│                    │
│  node-a ◄──► node-b│
│  Akka :9121 :9122  │
│  gRPC :9123 :9124  │
└────────────────────┘

Port Allocation

Env2 host ports are the primary's ports + 100. Both stacks can run simultaneously.

Node Container Name Host Web Host Akka Host gRPC Internal
Traefik LB scadabridge-env2-traefik 9100 — — 80 (proxy), 8080 (dashboard host:8181)
Central A scadabridge-env2-central-a 9101 9111 9113 5000 (web), 8081 (Akka), 8083 (gRPC)
Central B scadabridge-env2-central-b 9102 9112 9114 5000 (web), 8081 (Akka), 8083 (gRPC)
Site-X A scadabridge-env2-site-x-a — 9121 9123 8082 (Akka), 8083 (gRPC)
Site-X B scadabridge-env2-site-x-b — 9122 9124 8082 (Akka), 8083 (gRPC)

Shared Infrastructure

Env2 attaches to the existing scadabridge-net Docker bridge network and reuses these primary infra containers:

Service Container What env2 uses it for
MS SQL scadabridge-mssql Env2-specific databases ScadaBridgeConfig2 / ScadaBridgeMachineData2
SMTP scadabridge-smtp Notification capture in Mailpit (env2 emails distinguishable by FromAddress)
OPC UA scadabridge-opcua Simulated tags for site-x data connections
REST API scadabridge-restapi External REST API testing

LDAP is not one of the reused infra/ containers (that stack's ldap service is retired/commented out). Env2's central nodes bind to the same local redundant GLAuth pair as docker/ — zb-ldap-primary on host.docker.internal:3893 with zb-ldap-backup on host.docker.internal:3894 as FallbackServers (source of truth: scadaproj/infra/glauth-redundant); same test users, same bind credentials as the primary stack.

Commands

First-Time Setup

# 1. Make sure primary infra is up (creates scadabridge-net, scadabridge-mssql, etc.)
cd infra && docker compose up -d && cd ..

# 2. Build image + create env2 databases + deploy env2 containers
bash docker-env2/deploy.sh

# 3. Seed env2's single test site (first-time only)
bash docker-env2/seed-sites.sh

# 4. Optional: live data + store-and-forward load for site-x (opc-plc, a read-only
#    Galaxy attribute, and the scadabridge-env2-sfsink HTTP sink) — the #116 soak load
bash docker-env2/seed-data-path.sh

The MxGateway data connections need the windev gateway API key, which is not in the repo. Put it in ~/.config/scadabridge/windev-mxgw-apikey (mode 600), point MXGW_APIKEY_FILE at another file, or export MXGW_APIKEY. Without a key the script skips the MxGateway connection step and seeds everything else. The script never puts the key on a command line: it writes the gateway config to a mode-600 temp file (removed on exit by a trap) and passes it with data-connection create --primary-config-file (ScadaBridge#137; see Secrets from a file or stdin in src/ZB.MOM.WW.ScadaBridge.CLI/README.md).

Site-x's LocalDb is replicated (node-a dials node-b on 8083, same dev key on both), like the rig's site-a. Without it a site failover brings up an active node with no deployed instances, connections or S&F buffer.

The seed also creates a native alarm source demo: a MxAlarmDemo template with a GalaxyAlarms native alarm source bound to the shared MxGateway connection, plus a deployed MxAlarmDemo-1 instance on site-x. This exercises the read-only native alarm mirror end-to-end — mirrored alarms appear live in the Central UI Debug View with severity and condition badges. Verify the binding with:

scadabridge --url http://localhost:9100 template native-alarm-source list --template-id <id>

After Code Changes

bash docker-env2/deploy.sh

The Docker build is shared with the primary stack — if you've just run docker/deploy.sh, the env2 build hits a fully cached image.

Running Both Stacks Concurrently

bash docker/deploy.sh         # primary
bash docker-env2/deploy.sh    # env2

Both UIs are now reachable:

Own image tag, and the leader-auto-down soak (ScadaBridge#116)

SCADABRIDGE_IMAGE=<repo:tag> makes docker/build.sh tag that image and this compose file run it, so an env2 build never retags the scadabridge:latest the primary rig runs. Unset keeps the old shared-image behaviour.

env2 was the soak environment for the leader-auto-down rollout from 2026-11-12. Since 2026-11-15 leader-auto-down is the shipped default, and env2's mounted appsettings say so, so plain deploy.sh now runs it. The old overlay docker-compose.leader-auto-down.yml is a no-op. For a comparison run on Akka's AutoDowning, use docker-compose.auto-down.yml, switching both pairs together:

SCADABRIDGE_IMAGE=scadabridge:env2-116 docker compose \
  -f docker-env2/docker-compose.yml -f docker-env2/docker-compose.auto-down.yml \
  up -d --force-recreate --no-deps central-a central-b site-x-a site-x-b

Drill: DRILL_ENV=env2 DRILL_PAIR=central|site-x bash docker/failover-drill.sh. Record: docs/plans/2026-11-12-leader-auto-down-env2-drill.md.

Teardown

bash docker-env2/teardown.sh

Containers stop; node data lives in named Docker volumes (scadabridge-env2-*-data) and logs in node directories, both left in place. To also remove the persistent data:

# Destructive: deletes all env2 SQLite data (site config cache, S&F buffers, LocalDb).
docker volume rm $(docker volume ls -q --filter name=scadabridge-env2-)
rm -rf docker-env2/*/logs

To also drop the env2 databases:

docker exec scadabridge-mssql /opt/mssql-tools18/bin/sqlcmd \
    -S localhost -U sa -P 'ScadaBridge_Dev1#' -C \
    -Q "DROP DATABASE ScadaBridgeConfig2; DROP DATABASE ScadaBridgeMachineData2;"

CLI Access

dotnet run --project src/ZB.MOM.WW.ScadaBridge.CLI -- \
    --url http://localhost:9100 \
    --username multi-role --password password \
    template list

View Logs

docker compose -f docker-env2/docker-compose.yml logs -f
docker logs -f scadabridge-env2-central-a

Test Users

Same as primary (env2 shares the local redundant GLAuth pair, scadaproj/infra/glauth-redundant). See "Shared Infrastructure" above and primary docker/README.md.

Transport Testing Workflow

See docs/plans/2026-05-24-second-environment-verification.md for the manual golden-path checklist.

What's Different from Primary

  • Single site (site-x) instead of three (site-a/b/c).
  • Host port range 91XX vs primary 90XX.
  • Container names prefixed scadabridge-env2-.
  • Databases ScadaBridgeConfig2 / ScadaBridgeMachineData2 on the shared scadabridge-mssql.
  • Transport.SourceEnvironment = "docker-cluster-env2" (stamped into exported bundle manifests).
  • Distinct Security.JwtSigningKey (sessions cannot cross envs).