Joseph Doherty
2e4e41a8f7
Closes WP1.2 of the arch-review remediation plan (finding #2, High): SqliteAuditWriterOptions.DatabasePath defaulted to CWD-relative "auditlog.db", which on the docker rig resolves onto the container's ephemeral overlayfs (not the mounted /app/data volume), silently discarding the pending audit forward-state backlog on every recreate; nothing in docker/ or docker-env2/ overrode it; FlushIntervalMs was validated but never read by the writer loop (one commit per event even at trickle rate); and no PRAGMA synchronous was set (SQLite's FULL default fsyncs every commit). - DatabasePath now has no default (mirrors ZB.MOM.WW.LocalDb's LocalDbOptions.Path) and is required pre-host for Site nodes only, via a new StartupValidator raw-config check (top-level "AuditLog:SiteWriter:DatabasePath", NOT nested under ScadaBridge: AddAuditLog binds that section off the configuration root). SqliteAuditWriterOptionsValidator deliberately does NOT check DatabasePath itself, because AddAuditLog runs its ValidateOnStart on both Central and Site composition roots but only Site nodes ever resolve the writer — checking it there would fail Central's boot too. - All 8 site-node appsettings under docker/ and docker-env2/ now set AuditLog:SiteWriter:DatabasePath to /app/data/auditlog.db (mounted volume, survives container recreate, same convention as LocalDb:Path); the local-dev base appsettings.Site.json sets ./data/auditlog.db to match. - The writer loop now honors FlushIntervalMs: after draining the immediately available burst, it keeps the transaction open (bounded by FlushIntervalMs from the first event) waiting for more trickle-rate events before committing, instead of flushing (and fsyncing) per event. - PRAGMA synchronous = NORMAL on the write connection — audit is best-effort by design (CLAUDE.md: "Audit-write failure NEVER aborts the user-facing action"), so NORMAL's narrower power-loss window is an acceptable trade for far fewer fsyncs; WAL mode still guarantees no corruption. - Tests: StartupValidator site-required/blank/central-exempt cases; writer trickle-load single-transaction coalescing + beyond-interval separate-transaction regression (new FlushCountForTests seam); options-validator doc updates reflecting the moved responsibility. Full suite runs green: AuditLog.Tests 368/368, Host.Tests 480/480. One-time migration note: the existing container-local auditlog.db (wherever it landed under CWD) is abandoned by this change, not migrated — already-forwarded rows are safe centrally (AuditLog is the durable copy), and any still-Pending rows on the abandoned path are lost once. This is the exact bug being fixed, not a new loss: those rows were already living outside the mounted volume and would not have survived the next container recreate regardless. Cross-reference docs/known-issues/2026-07-20-cached-telemetry-drain-hot-loop.md, which this placement bug caused.
ScadaBridge Env2 Docker Infrastructure
A second Docker deployment of a minimal ScadaBridge cluster topology, designed to run concurrently with the primary docker/ stack so the Transport (#24) feature can be exercised end-to-end across two real environments.
See docs/plans/2026-05-24-second-environment-design.md for the design rationale.
Cluster Topology
┌───────────────────┐
│ Traefik LB :9100 │ ◄── CLI / Browser
│ Dashboard :8181 │
└────────┬──────────┘
│ routes to active node
┌──────────────────────┼──────────────────────────────┐
│ Env2 Central Cluster │
│ │
│ ┌─────────────────┐ ┌─────────────────┐ │
│ │ env2-central-a │◄──►│ env2-central-b │ │
│ │ Web UI :9101 │ │ Web UI :9102 │ │
│ │ Akka :9111 │ │ Akka :9112 │ │
│ └────────┬─────────┘ └─────────────────┘ │
│ │ │
└───────────┼─────────────────────────────────────────┘
│ Akka.NET Remoting
▼
┌────────────────────┐
│ Env2 Site-X │
│ (Env2 Site X) │
│ │
│ node-a ◄──► node-b│
│ Akka :9121 :9122 │
│ gRPC :9123 :9124 │
└────────────────────┘
Port Allocation
Env2 host ports are the primary's ports + 100. Both stacks can run simultaneously.
| Node | Container Name | Host Web | Host Akka | Host gRPC | Internal |
|---|---|---|---|---|---|
| Traefik LB | scadabridge-env2-traefik |
9100 | — | — | 80 (proxy), 8080 (dashboard host:8181) |
| Central A | scadabridge-env2-central-a |
9101 | 9111 | — | 5000 (web), 8081 (Akka) |
| Central B | scadabridge-env2-central-b |
9102 | 9112 | — | 5000 (web), 8081 (Akka) |
| Site-X A | scadabridge-env2-site-x-a |
— | 9121 | 9123 | 8082 (Akka), 8083 (gRPC) |
| Site-X B | scadabridge-env2-site-x-b |
— | 9122 | 9124 | 8082 (Akka), 8083 (gRPC) |
Shared Infrastructure
Env2 attaches to the existing scadabridge-net Docker bridge network and reuses these primary infra containers:
| Service | Container | What env2 uses it for |
|---|---|---|
| MS SQL | scadabridge-mssql |
Env2-specific databases ScadaBridgeConfig2 / ScadaBridgeMachineData2 |
| LDAP | scadabridge-ldap |
Authentication (same test users) |
| SMTP | scadabridge-smtp |
Notification capture in Mailpit (env2 emails distinguishable by FromAddress) |
| OPC UA | scadabridge-opcua |
Simulated tags for site-x data connections |
| REST API | scadabridge-restapi |
External REST API testing |
Commands
First-Time Setup
# 1. Make sure primary infra is up (creates scadabridge-net, scadabridge-mssql, etc.)
cd infra && docker compose up -d && cd ..
# 2. Build image + create env2 databases + deploy env2 containers
bash docker-env2/deploy.sh
# 3. Seed env2's single test site (first-time only)
bash docker-env2/seed-sites.sh
The seed also creates a native alarm source demo: a MxAlarmDemo template with a
GalaxyAlarms native alarm source bound to the shared MxGateway connection, plus a
deployed MxAlarmDemo-1 instance on site-x. This exercises the read-only native alarm
mirror end-to-end — mirrored alarms appear live in the Central UI Debug View with severity
and condition badges. Verify the binding with:
scadabridge --url http://localhost:9100 template native-alarm-source list --template-id <id>
After Code Changes
bash docker-env2/deploy.sh
The Docker build is shared with the primary stack — if you've just run docker/deploy.sh, the env2 build hits a fully cached image.
Running Both Stacks Concurrently
bash docker/deploy.sh # primary
bash docker-env2/deploy.sh # env2
Both UIs are now reachable:
- Primary: http://localhost:9000
- Env2: http://localhost:9100
Teardown
bash docker-env2/teardown.sh
Containers stop, volumes (data + logs) preserved. To also drop the env2 databases:
docker exec scadabridge-mssql /opt/mssql-tools18/bin/sqlcmd \
-S localhost -U sa -P 'ScadaBridge_Dev1#' -C \
-Q "DROP DATABASE ScadaBridgeConfig2; DROP DATABASE ScadaBridgeMachineData2;"
CLI Access
dotnet run --project src/ZB.MOM.WW.ScadaBridge.CLI -- \
--url http://localhost:9100 \
--username multi-role --password password \
template list
View Logs
docker compose -f docker-env2/docker-compose.yml logs -f
docker logs -f scadabridge-env2-central-a
Test Users
Same as primary (env2 shares LDAP). See infra/glauth/config.toml and primary docker/README.md.
Transport Testing Workflow
See docs/plans/2026-05-24-second-environment-verification.md for the manual golden-path checklist.
What's Different from Primary
- Single site (
site-x) instead of three (site-a/b/c). - Host port range 91XX vs primary 90XX.
- Container names prefixed
scadabridge-env2-. - Databases
ScadaBridgeConfig2/ScadaBridgeMachineData2on the sharedscadabridge-mssql. Transport.SourceEnvironment="docker-cluster-env2"(stamped into exported bundle manifests).- Distinct
Security.JwtSigningKey(sessions cannot cross envs).