Joseph Doherty
e2f433e849
Every CLI option that can carry a secret gets a `--<name>-file <path|->`
sibling ('-' reads stdin), so the value stays out of the process list and
shell history: --password-file (global), --primary-config-file /
--backup-config-file (data-connection create|update), --auth-config-file
(external-system create|update), --connection-string-file (db-connection
create|update), --credentials-file (notification smtp create|update) and
--auth-token-file (notification sms create|update). Follows the existing
--config-file convention; `@path` is avoided because System.CommandLine
expands @-tokens as response files.
The value is read at parse time; a missing/unreadable/empty source, or '-'
on a terminal stdin, is a parse error naming option + path only, never the
content. Inline and file forms are mutually exclusive, at most one option
may read stdin, and one trailing line break is stripped. Inline forms keep
working.
docker/seed-sites.sh and docker-env2/seed-sites.sh now write the MxGateway
config (windev gateway key) to a mode-600 temp file removed by an EXIT trap
and pass --primary-config-file; seed-flow-outbound-demo.sh pipes its rig
connection string via --connection-string-file -. CLI README (new "Secrets
from a file or stdin" section + per-command tables), Component-CLI.md and
both docker READMEs updated.
Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
203 lines
8.9 KiB
Markdown
203 lines
8.9 KiB
Markdown
# ScadaBridge Env2 Docker Infrastructure
|
|
|
|
A second Docker deployment of a minimal ScadaBridge cluster topology, designed to run **concurrently with** the primary `docker/` stack so the Transport (#24) feature can be exercised end-to-end across two real environments.
|
|
|
|
See [`docs/plans/2026-05-24-second-environment-design.md`](../docs/plans/2026-05-24-second-environment-design.md) for the design rationale.
|
|
|
|
## Cluster Topology
|
|
|
|
```
|
|
┌───────────────────┐
|
|
│ Traefik LB :9100 │ ◄── CLI / Browser
|
|
│ Dashboard :8181 │
|
|
└────────┬──────────┘
|
|
│ routes to active node
|
|
┌──────────────────────┼──────────────────────────────┐
|
|
│ Env2 Central Cluster │
|
|
│ │
|
|
│ ┌─────────────────┐ ┌─────────────────┐ │
|
|
│ │ env2-central-a │◄──►│ env2-central-b │ │
|
|
│ │ Web UI :9101 │ │ Web UI :9102 │ │
|
|
│ │ Akka :9111 │ │ Akka :9112 │ │
|
|
│ └────────┬─────────┘ └─────────────────┘ │
|
|
│ │ │
|
|
└───────────┼─────────────────────────────────────────┘
|
|
│ Akka.NET Remoting
|
|
▼
|
|
┌────────────────────┐
|
|
│ Env2 Site-X │
|
|
│ (Env2 Site X) │
|
|
│ │
|
|
│ node-a ◄──► node-b│
|
|
│ Akka :9121 :9122 │
|
|
│ gRPC :9123 :9124 │
|
|
└────────────────────┘
|
|
```
|
|
|
|
## Port Allocation
|
|
|
|
Env2 host ports are the primary's ports + 100. Both stacks can run simultaneously.
|
|
|
|
| Node | Container Name | Host Web | Host Akka | Host gRPC | Internal |
|
|
|------|----------------|----------|-----------|-----------|----------|
|
|
| Traefik LB | `scadabridge-env2-traefik` | 9100 | — | — | 80 (proxy), 8080 (dashboard host:8181) |
|
|
| Central A | `scadabridge-env2-central-a` | 9101 | 9111 | 9113 | 5000 (web), 8081 (Akka), 8083 (gRPC) |
|
|
| Central B | `scadabridge-env2-central-b` | 9102 | 9112 | 9114 | 5000 (web), 8081 (Akka), 8083 (gRPC) |
|
|
| Site-X A | `scadabridge-env2-site-x-a` | — | 9121 | 9123 | 8082 (Akka), 8083 (gRPC) |
|
|
| Site-X B | `scadabridge-env2-site-x-b` | — | 9122 | 9124 | 8082 (Akka), 8083 (gRPC) |
|
|
|
|
## Shared Infrastructure
|
|
|
|
Env2 attaches to the existing `scadabridge-net` Docker bridge network and reuses these primary infra containers:
|
|
|
|
| Service | Container | What env2 uses it for |
|
|
|---------|-----------|-----------------------|
|
|
| MS SQL | `scadabridge-mssql` | Env2-specific databases `ScadaBridgeConfig2` / `ScadaBridgeMachineData2` |
|
|
| SMTP | `scadabridge-smtp` | Notification capture in Mailpit (env2 emails distinguishable by `FromAddress`) |
|
|
| OPC UA | `scadabridge-opcua` | Simulated tags for site-x data connections |
|
|
| REST API | `scadabridge-restapi` | External REST API testing |
|
|
|
|
LDAP is **not** one of the reused `infra/` containers (that stack's `ldap` service is
|
|
retired/commented out). Env2's central nodes bind to the same local redundant GLAuth pair
|
|
as `docker/` — `zb-ldap-primary` on `host.docker.internal:3893` with `zb-ldap-backup` on
|
|
`host.docker.internal:3894` as `FallbackServers` (source of truth:
|
|
`scadaproj/infra/glauth-redundant`); same test users, same bind credentials as the primary
|
|
stack.
|
|
|
|
## Commands
|
|
|
|
### First-Time Setup
|
|
|
|
```bash
|
|
# 1. Make sure primary infra is up (creates scadabridge-net, scadabridge-mssql, etc.)
|
|
cd infra && docker compose up -d && cd ..
|
|
|
|
# 2. Build image + create env2 databases + deploy env2 containers
|
|
bash docker-env2/deploy.sh
|
|
|
|
# 3. Seed env2's single test site (first-time only)
|
|
bash docker-env2/seed-sites.sh
|
|
|
|
# 4. Optional: live data + store-and-forward load for site-x (opc-plc, a read-only
|
|
# Galaxy attribute, and the scadabridge-env2-sfsink HTTP sink) — the #116 soak load
|
|
bash docker-env2/seed-data-path.sh
|
|
```
|
|
|
|
> The MxGateway data connections need the windev gateway API key, which is not in the repo. Put it in
|
|
> `~/.config/scadabridge/windev-mxgw-apikey` (mode 600), point `MXGW_APIKEY_FILE` at another file, or export
|
|
> `MXGW_APIKEY`. Without a key the script skips the MxGateway connection step and seeds everything else.
|
|
> The script never puts the key on a command line: it writes the gateway config to a mode-600 temp file
|
|
> (removed on exit by a trap) and passes it with `data-connection create --primary-config-file`
|
|
> (ScadaBridge#137; see *Secrets from a file or stdin* in `src/ZB.MOM.WW.ScadaBridge.CLI/README.md`).
|
|
|
|
Site-x's LocalDb is **replicated** (node-a dials node-b on 8083, same dev key on both),
|
|
like the rig's site-a. Without it a site failover brings up an active node with no
|
|
deployed instances, connections or S&F buffer.
|
|
|
|
The seed also creates a **native alarm source demo**: a `MxAlarmDemo` template with a
|
|
`GalaxyAlarms` native alarm source bound to the shared MxGateway connection, plus a
|
|
deployed `MxAlarmDemo-1` instance on site-x. This exercises the read-only native alarm
|
|
mirror end-to-end — mirrored alarms appear live in the Central UI Debug View with severity
|
|
and condition badges. Verify the binding with:
|
|
|
|
```bash
|
|
scadabridge --url http://localhost:9100 template native-alarm-source list --template-id <id>
|
|
```
|
|
|
|
### After Code Changes
|
|
|
|
```bash
|
|
bash docker-env2/deploy.sh
|
|
```
|
|
|
|
The Docker build is shared with the primary stack — if you've just run `docker/deploy.sh`, the env2 build hits a fully cached image.
|
|
|
|
### Running Both Stacks Concurrently
|
|
|
|
```bash
|
|
bash docker/deploy.sh # primary
|
|
bash docker-env2/deploy.sh # env2
|
|
```
|
|
|
|
Both UIs are now reachable:
|
|
- Primary: http://localhost:9000
|
|
- Env2: http://localhost:9100
|
|
|
|
### Own image tag, and the `leader-auto-down` soak (ScadaBridge#116)
|
|
|
|
`SCADABRIDGE_IMAGE=<repo:tag>` makes `docker/build.sh` tag that image and this compose file
|
|
run it, so an env2 build never retags the `scadabridge:latest` the primary rig runs. Unset
|
|
keeps the old shared-image behaviour.
|
|
|
|
Since 2026-11-12 env2 is the soak environment for the `leader-auto-down` rollout and runs
|
|
`scadabridge:env2-116` with the opt-in overlay
|
|
[`docker-compose.leader-auto-down.yml`](docker-compose.leader-auto-down.yml). Keep both
|
|
when recreating nodes, and switch both pairs together:
|
|
|
|
```bash
|
|
SCADABRIDGE_IMAGE=scadabridge:env2-116 docker compose \
|
|
-f docker-env2/docker-compose.yml -f docker-env2/docker-compose.leader-auto-down.yml \
|
|
up -d --force-recreate --no-deps central-a central-b site-x-a site-x-b
|
|
```
|
|
|
|
Plain `deploy.sh` goes back to `scadabridge:latest` on the default `auto-down`. Drill:
|
|
`DRILL_ENV=env2 DRILL_PAIR=central|site-x bash docker/failover-drill.sh`. Record:
|
|
[`docs/plans/2026-11-12-leader-auto-down-env2-drill.md`](../docs/plans/2026-11-12-leader-auto-down-env2-drill.md).
|
|
|
|
### Teardown
|
|
|
|
```bash
|
|
bash docker-env2/teardown.sh
|
|
```
|
|
|
|
Containers stop; node data lives in named Docker volumes (`scadabridge-env2-*-data`) and
|
|
logs in node directories, both left in place. To also remove the persistent data:
|
|
|
|
```bash
|
|
# Destructive: deletes all env2 SQLite data (site config cache, S&F buffers, LocalDb).
|
|
docker volume rm $(docker volume ls -q --filter name=scadabridge-env2-)
|
|
rm -rf docker-env2/*/logs
|
|
```
|
|
|
|
To also drop the env2 databases:
|
|
|
|
```bash
|
|
docker exec scadabridge-mssql /opt/mssql-tools18/bin/sqlcmd \
|
|
-S localhost -U sa -P 'ScadaBridge_Dev1#' -C \
|
|
-Q "DROP DATABASE ScadaBridgeConfig2; DROP DATABASE ScadaBridgeMachineData2;"
|
|
```
|
|
|
|
### CLI Access
|
|
|
|
```bash
|
|
dotnet run --project src/ZB.MOM.WW.ScadaBridge.CLI -- \
|
|
--url http://localhost:9100 \
|
|
--username multi-role --password password \
|
|
template list
|
|
```
|
|
|
|
### View Logs
|
|
|
|
```bash
|
|
docker compose -f docker-env2/docker-compose.yml logs -f
|
|
docker logs -f scadabridge-env2-central-a
|
|
```
|
|
|
|
## Test Users
|
|
|
|
Same as primary (env2 shares the local redundant GLAuth pair, `scadaproj/infra/glauth-redundant`).
|
|
See "Shared Infrastructure" above and primary `docker/README.md`.
|
|
|
|
## Transport Testing Workflow
|
|
|
|
See [`docs/plans/2026-05-24-second-environment-verification.md`](../docs/plans/2026-05-24-second-environment-verification.md) for the manual golden-path checklist.
|
|
|
|
## What's Different from Primary
|
|
|
|
- Single site (`site-x`) instead of three (`site-a/b/c`).
|
|
- Host port range 91XX vs primary 90XX.
|
|
- Container names prefixed `scadabridge-env2-`.
|
|
- Databases `ScadaBridgeConfig2` / `ScadaBridgeMachineData2` on the **shared** `scadabridge-mssql`.
|
|
- `Transport.SourceEnvironment` = `"docker-cluster-env2"` (stamped into exported bundle manifests).
|
|
- Distinct `Security.JwtSigningKey` (sessions cannot cross envs).
|