Joseph Doherty
8652eab98e
The Phase 2 soak's "LocalDb fails under load" blocker is NOT a product defect. Root cause: host-side (macOS) sqlite3 reads of the live, bind-mounted WAL databases. POSIX advisory locks do not propagate across the virtiofs boundary, so the host reader believes it is the only connection, checkpoints on close and resets the WAL to 0 bytes under the container. The container's still-mapped WAL index then references frames that no longer exist and every subsequent statement fails SQLITE_IOERR_SHORT_READ (522) -> primary code 10, permanently until the process reopens the database. Reproduced on demand both on the rig (one sqlite3 SELECT reset a 4.6 MiB WAL and produced the first error one second later) and in a minimal python:3.12-alpine repro with no LocalDb or .NET involved. Refuted the converse: a freshly-reopened node sustained the full soak load 10+ minutes with zero errors. The original brief's isolation was confounded - both nodes had been poisoned by the same sampling pass, and a poisoned standby looks healthy only because it issues almost no statements. Corrections annotated in place. Hardening shipped alongside: - SqliteErrorCodes.Describe: log SQLite primary AND extended codes at the LocalDb-adjacent catch sites (the missing extended code is what made the original diagnosis so slow). - SiteAuditTelemetryActor: stop touching ActorContext across an await (NotSupportedException), with regression coverage. - infra/reseed.sh: apply the MSSQL init scripts explicitly. The official mssql/server image does not implement /docker-entrypoint-initdb.d, so a fresh volume hung the reseed forever; compose mounts annotated as informational. Deliberately NOT done: detect-and-reopen self-heal in SqliteLocalDb. It defends only against external interference, which is now prevented at the source, and same-kernel production readers see the locks correctly. Build 0 warnings; SiteAuditTelemetryActorTests 9/9, SiteEventLogging 70/70. Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts
ScadaBridge Test Infrastructure
Local Docker-based test services for ScadaBridge development.
Quick Start
docker compose up -d
This starts the following services:
| Service | Port | Purpose |
|---|---|---|
| OPC UA (Azure IoT OPC PLC) | 50000 (OPC UA), 8080 (web) | Simulated OPC UA server with ScadaBridge-style tags |
| OPC UA 2 (Azure IoT OPC PLC) | 50010 (OPC UA), 8081 (web) | Second OPC UA server instance (same tags, independent state) |
RETIRED (2026-06-04) — no longer started by this stack. Dev/test LDAP is the shared GLAuth on 10.100.0.35:3893 (dc=zb,dc=local). Central nodes bind Ldap:Server=10.100.0.35. Source of truth + config: scadaproj/infra/glauth/. |
||
| MS SQL 2022 | 1433 | Configuration and machine data databases |
| SMTP (Mailpit) | 1025 (SMTP), 8025 (web) | Email capture for notification testing |
| REST API (Flask) | 5200 | External REST API for Gateway and Inbound API testing |
First-Time SQL Setup
The MS SQL container does not auto-run init scripts. After the first docker compose up -d, run:
docker exec -i scadabridge-mssql /opt/mssql-tools18/bin/sqlcmd \
-S localhost -U sa -P 'ScadaBridge_Dev1#' -C \
-i /docker-entrypoint-initdb.d/setup.sql
This creates the ScadaBridgeConfig and ScadaBridgeMachineData databases and the scadabridge_app login. Then seed the Machine Data database:
docker exec -i scadabridge-mssql /opt/mssql-tools18/bin/sqlcmd \
-S localhost -U sa -P 'ScadaBridge_Dev1#' -C \
-i /docker-entrypoint-initdb.d/machinedata_seed.sql
For the second environment (docker-env2/), also apply the env2 database setup:
docker exec -i scadabridge-mssql /opt/mssql-tools18/bin/sqlcmd \
-S localhost -U sa -P 'ScadaBridge_Dev1#' -C \
-i /docker-entrypoint-initdb.d/setup-env2.sql
This creates ScadaBridgeConfig2 and ScadaBridgeMachineData2 databases on the same MSSQL instance. The script is also invoked automatically by docker-env2/deploy.sh via docker-env2/init-db.sh, so manual application here is only needed if you want the databases ready before first env2 deploy.
Stopping & Teardown
Stop containers (data persists in SQL volume):
docker compose down
Stop a single service (leave the others running):
docker compose stop opcua # or: opcua2, mssql, smtp, restapi (ldap is no longer a local service)
docker compose start opcua # bring it back without recreating
Full teardown (stop containers, delete SQL data volume, remove pulled images):
./teardown.sh
Or manually:
docker compose down -v # stop containers + delete SQL data volume
docker compose down -v --rmi all # also remove downloaded images
After a full teardown, the next docker compose up -d starts fresh — you'll need to re-run the SQL setup script.
CLI Tools
Python CLI tools for interacting with the test services are in tools/.
Set up a Python virtual environment (one-time):
python3 -m venv tools/.venv && source tools/.venv/bin/activate
Install dependencies (one-time, with venv active):
pip install -r tools/requirements.txt
The
.venvdirectory is gitignored.
Quick readiness check (all five services, with venv active):
python tools/opcua_tool.py check
python tools/ldap_tool.py check
python tools/mssql_tool.py check
python tools/smtp_tool.py check
python tools/restapi_tool.py check
| Tool | Service | Key Commands |
|---|---|---|
tools/opcua_tool.py |
OPC UA | check, browse, read, write, monitor |
tools/ldap_tool.py |
LDAP | check, bind, search, users, groups |
tools/mssql_tool.py |
MS SQL | check, setup, query, tables |
tools/smtp_tool.py |
SMTP (Mailpit) | check, send, list, read, clear |
tools/restapi_tool.py |
REST API (Flask) | check, call, methods |
Each tool supports --help for full usage. See the per-service docs below for detailed examples.
Detailed Documentation
See docs/test_infra/ for per-service setup guides:
- test_infra.md — Master test infrastructure overview
- test_infra_opcua.md — OPC UA server details
- test_infra_ldap.md — LDAP server details
- test_infra_db.md — MS SQL database details
- test_infra_smtp.md — SMTP server details (Mailpit)
- test_infra_restapi.md — REST API server details (Flask)