infra(docker)+test(siteeventlog): close register tripwires 79 + 93 — rig data to named volumes; pin site_events schema to the cap-model calibration

Row 79 executed its own prescribed remedy without waiting for a second corruption: all ten rig data dirs (8 docker/ nodes + 2 docker-env2/ site nodes) move from virtiofs bind mounts to named docker volumes (scadabridge-<node>-data), seeded in place with both rigs stopped and live-verified (site-a localDbReplicationConnected=true / backlog 0 over the migrated files; env2 stable, 0 restarts). The feared inspectability loss is moot — OrbStack exposes volumes at ~/OrbStack/docker/volumes/. The rebuild surfaced latent env2 drift (pre-Secrets-0.6.2 image): env2 now carries the x-secrets-store-env posture + central data volumes. Row 93's silent maintenance trap is now loud: SiteEventsSchemaCalibrationTests pins the site_events columns/indexes the EventLogPurgeService size model is calibrated against, failing with a pointer to the memo §4 recalibration on any schema drift. Rows 92 (accepted by design) and 94 (standing tripwire) annotated accordingly.
2026-08-18 08:43:52 -04:00
parent 5936573e71
commit 324ba369d9
8 changed files with 284 additions and 83 deletions
@@ -97,30 +97,29 @@ docker/
 │   └── logs/
 ├── site-a-node-a/
 │   ├── appsettings.Site.json           # Site-A node A configuration
-│   ├── data/                           # SQLite databases (gitignored)
-│   └── logs/
+│   └── logs/                           # Serilog file output (gitignored)
 ├── site-a-node-b/
 │   ├── appsettings.Site.json
-│   ├── data/
 │   └── logs/
 ├── site-b-node-a/
 │   ├── appsettings.Site.json           # Site-B node A configuration
-│   ├── data/
 │   └── logs/
 ├── site-b-node-b/
 │   ├── appsettings.Site.json
-│   ├── data/
 │   └── logs/
 ├── site-c-node-a/
 │   ├── appsettings.Site.json           # Site-C node A configuration
-│   ├── data/
 │   └── logs/
 └── site-c-node-b/
    ├── appsettings.Site.json
-    ├── data/
    └── logs/
 ```

+SQLite databases live in **named docker volumes** (`scadabridge-<node>-data`, one per node —
+register row 79), not in the node directories; inspect them from the host (read-only) at
+`~/OrbStack/docker/volumes/<name>/`. Any `docker/<node>/data/` directory still on disk is a
+dormant pre-migration (2026-08-18) backup, no longer mounted.
+
 ## gRPC control-plane keys (dev)

 The site gRPC service (`SiteStreamService` on 8083 — live subscriptions, audit pull,
@@ -315,20 +314,27 @@ dotnet run --project src/ZB.MOM.WW.ScadaBridge.CLI -- \

 ### Clear Site Data

-Remove SQLite databases to reset site state (deployed configs, S&F buffers):
+Remove SQLite databases to reset site state (deployed configs, S&F buffers). Data lives in
+named docker volumes (`scadabridge-<node>-data`, register row 79 — no longer bind-mounted
+under `docker/<node>/data`), so empty the volume with a throwaway container while the pair
+is stopped:

 ```bash
 # Single site
-rm -rf docker/site-a-node-a/data docker/site-a-node-b/data
-docker restart scadabridge-site-a-a scadabridge-site-a-b
+docker compose -f docker/docker-compose.yml stop site-a-a site-a-b
+docker run --rm \
+    -v scadabridge-site-a-node-a-data:/a -v scadabridge-site-a-node-b-data:/b \
+    alpine sh -c 'rm -rf /a/* /b/*'
+docker compose -f docker/docker-compose.yml start site-a-a site-a-b

-# All sites
-rm -rf docker/site-*/data
-docker restart scadabridge-site-a-a scadabridge-site-a-b \
-    scadabridge-site-b-a scadabridge-site-b-b \
-    scadabridge-site-c-a scadabridge-site-c-b
+# All sites: repeat with the six scadabridge-site-*-data volumes, or teardown + volume rm
+# (see teardown.sh's hint) and redeploy.
 ```

+To inspect a node's live databases from the host, read them via OrbStack's volume
+exposure (read-only inspection — don't write through this path):
+`~/OrbStack/docker/volumes/scadabridge-site-a-node-a-data/site-localdb.db`.
+
 ### Rebuild Image From Scratch (no cache)

 ```bash
@@ -3,13 +3,14 @@ set -euo pipefail

 SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"

-# The compose file's bind mounts are RELATIVE (./site-a-node-a/data etc.), so running
-# this script from a git worktree silently rehomes every node's data directory to the
-# worktree's copy — the rig keeps running, but its SQLite stores split from the main
-# checkout's and all state written there is lost on the next deploy from main.
+# The compose file's remaining bind mounts (appsettings, logs) are RELATIVE, so running
+# this script from a git worktree silently rehomes every node's config to the worktree's
+# copy — the rig keeps running, but on stale/divergent appsettings. (The data dirs moved
+# to named volumes — register row 79 — so rig state itself no longer splits, but a
+# worktree deploy is still wrong by default.)
 if [[ "$SCRIPT_DIR" == *"/.claude/worktrees/"* && "${SCADABRIDGE_ALLOW_WORKTREE_DEPLOY:-}" != "1" ]]; then
  echo "ERROR: refusing to deploy from a git worktree ($SCRIPT_DIR)." >&2
-  echo "Bind-mount data dirs would rehome to the worktree copy and split rig state." >&2
+  echo "Bind-mounted appsettings would rehome to the worktree copy." >&2
  echo "Deploy from the main checkout, or set SCADABRIDGE_ALLOW_WORKTREE_DEPLOY=1 to override." >&2
  exit 1
 fi
@@ -6,9 +6,9 @@
 # The value must be absolute AND outside the content root (/app) — Secrets 0.6.2 validates
 # both at startup, because a store inside the deployment directory is destroyed by an
 # ordinary upgrade (this wiped the MxGateway key store on 2026-08-09). /data is the node's
-# own bind mount, the SAME host directory as /app/data, so an existing store is found
-# unmoved; only the path used to reach it changes. Do NOT point this back at /app/data —
-# absolute is not sufficient, and the boot will fail.
+# own data volume (named volume since register row 79), the SAME volume as /app/data, so an
+# existing store is found unmoved; only the path used to reach it changes. Do NOT point
+# this back at /app/data — absolute is not sufficient, and the boot will fail.
 #
 # Do NOT delete this key expecting the library's per-user default to take over. Measured on
 # this rig: with a Secrets section present but no SqlitePath, the effective value is the
@@ -129,11 +129,11 @@ services:
      # central store moved to the shared SQL-Server database (Secrets 0.5.0,
      # scadaproj#4) the scadabridge-secrets.db here is a pre-0.5.0 residue, but the
      # volume is still needed (inbound-api-keys.sqlite lives on it).
-      - ./central-node-a/data:/app/data
-      # Same host directory, mounted a second time OUTSIDE the content root (/app).
-      # Secrets 0.6.2 refuses a store path under the content root; this bind mount is
+      - central-node-a-data:/app/data
+      # Same named volume, mounted a second time OUTSIDE the content root (/app).
+      # Secrets 0.6.2 refuses a store path under the content root; this mount is
      # not touched by an image rebuild, but the guard is lexical and cannot know that.
-      - ./central-node-a/data:/data
+      - central-node-a-data:/data
      - ./central-node-a/logs:/app/logs
    networks:
      - scadabridge-net
@@ -190,11 +190,11 @@ services:
      # central store moved to the shared SQL-Server database (Secrets 0.5.0,
      # scadaproj#4) the scadabridge-secrets.db here is a pre-0.5.0 residue, but the
      # volume is still needed (inbound-api-keys.sqlite lives on it).
-      - ./central-node-b/data:/app/data
-      # Same host directory, mounted a second time OUTSIDE the content root (/app).
-      # Secrets 0.6.2 refuses a store path under the content root; this bind mount is
+      - central-node-b-data:/app/data
+      # Same named volume, mounted a second time OUTSIDE the content root (/app).
+      # Secrets 0.6.2 refuses a store path under the content root; this mount is
      # not touched by an image rebuild, but the guard is lexical and cannot know that.
-      - ./central-node-b/data:/data
+      - central-node-b-data:/data
      - ./central-node-b/logs:/app/logs
    networks:
      - scadabridge-net
@@ -226,11 +226,11 @@ services:
      - "9023:8083"   # gRPC streaming
    volumes:
      - ./site-a-node-a/appsettings.Site.json:/app/appsettings.Site.json:ro
-      - ./site-a-node-a/data:/app/data
-      # Same host directory, mounted a second time OUTSIDE the content root (/app).
-      # Secrets 0.6.2 refuses a store path under the content root; this bind mount is
+      - site-a-node-a-data:/app/data
+      # Same named volume, mounted a second time OUTSIDE the content root (/app).
+      # Secrets 0.6.2 refuses a store path under the content root; this mount is
      # not touched by an image rebuild, but the guard is lexical and cannot know that.
-      - ./site-a-node-a/data:/data
+      - site-a-node-a-data:/data
      - ./site-a-node-a/logs:/app/logs
    networks:
      - scadabridge-net
@@ -262,11 +262,11 @@ services:
      - "9024:8083"   # gRPC streaming
    volumes:
      - ./site-a-node-b/appsettings.Site.json:/app/appsettings.Site.json:ro
-      - ./site-a-node-b/data:/app/data
-      # Same host directory, mounted a second time OUTSIDE the content root (/app).
-      # Secrets 0.6.2 refuses a store path under the content root; this bind mount is
+      - site-a-node-b-data:/app/data
+      # Same named volume, mounted a second time OUTSIDE the content root (/app).
+      # Secrets 0.6.2 refuses a store path under the content root; this mount is
      # not touched by an image rebuild, but the guard is lexical and cannot know that.
-      - ./site-a-node-b/data:/data
+      - site-a-node-b-data:/data
      - ./site-a-node-b/logs:/app/logs
    networks:
      - scadabridge-net
@@ -296,11 +296,11 @@ services:
      - "9033:8083"   # gRPC streaming
    volumes:
      - ./site-b-node-a/appsettings.Site.json:/app/appsettings.Site.json:ro
-      - ./site-b-node-a/data:/app/data
-      # Same host directory, mounted a second time OUTSIDE the content root (/app).
-      # Secrets 0.6.2 refuses a store path under the content root; this bind mount is
+      - site-b-node-a-data:/app/data
+      # Same named volume, mounted a second time OUTSIDE the content root (/app).
+      # Secrets 0.6.2 refuses a store path under the content root; this mount is
      # not touched by an image rebuild, but the guard is lexical and cannot know that.
-      - ./site-b-node-a/data:/data
+      - site-b-node-a-data:/data
      - ./site-b-node-a/logs:/app/logs
    networks:
      - scadabridge-net
@@ -330,11 +330,11 @@ services:
      - "9034:8083"   # gRPC streaming
    volumes:
      - ./site-b-node-b/appsettings.Site.json:/app/appsettings.Site.json:ro
-      - ./site-b-node-b/data:/app/data
-      # Same host directory, mounted a second time OUTSIDE the content root (/app).
-      # Secrets 0.6.2 refuses a store path under the content root; this bind mount is
+      - site-b-node-b-data:/app/data
+      # Same named volume, mounted a second time OUTSIDE the content root (/app).
+      # Secrets 0.6.2 refuses a store path under the content root; this mount is
      # not touched by an image rebuild, but the guard is lexical and cannot know that.
-      - ./site-b-node-b/data:/data
+      - site-b-node-b-data:/data
      - ./site-b-node-b/logs:/app/logs
    networks:
      - scadabridge-net
@@ -364,11 +364,11 @@ services:
      - "9043:8083"   # gRPC streaming
    volumes:
      - ./site-c-node-a/appsettings.Site.json:/app/appsettings.Site.json:ro
-      - ./site-c-node-a/data:/app/data
-      # Same host directory, mounted a second time OUTSIDE the content root (/app).
-      # Secrets 0.6.2 refuses a store path under the content root; this bind mount is
+      - site-c-node-a-data:/app/data
+      # Same named volume, mounted a second time OUTSIDE the content root (/app).
+      # Secrets 0.6.2 refuses a store path under the content root; this mount is
      # not touched by an image rebuild, but the guard is lexical and cannot know that.
-      - ./site-c-node-a/data:/data
+      - site-c-node-a-data:/data
      - ./site-c-node-a/logs:/app/logs
    networks:
      - scadabridge-net
@@ -398,11 +398,11 @@ services:
      - "9044:8083"   # gRPC streaming
    volumes:
      - ./site-c-node-b/appsettings.Site.json:/app/appsettings.Site.json:ro
-      - ./site-c-node-b/data:/app/data
-      # Same host directory, mounted a second time OUTSIDE the content root (/app).
-      # Secrets 0.6.2 refuses a store path under the content root; this bind mount is
+      - site-c-node-b-data:/app/data
+      # Same named volume, mounted a second time OUTSIDE the content root (/app).
+      # Secrets 0.6.2 refuses a store path under the content root; this mount is
      # not touched by an image rebuild, but the guard is lexical and cannot know that.
-      - ./site-c-node-b/data:/data
+      - site-c-node-b-data:/data
      - ./site-c-node-b/logs:/app/logs
    networks:
      - scadabridge-net
@@ -424,3 +424,29 @@ services:
 networks:
  scadabridge-net:
    external: true
+
+# Named volumes for every node's SQLite data dir (register row 79). On macOS/OrbStack a
+# bind mount crosses virtiofs, which does not give SQLite true fsync durability — both
+# site-a LocalDb files corrupted through that layer (row 69). Named volumes live on
+# native ext4 inside the OrbStack VM, taking virtiofs out of the write path entirely.
+# Host-side inspection still works read-only via ~/OrbStack/docker/volumes/<name>;
+# `docker compose down` preserves them; remove with `docker volume rm <name>`.
+# The appsettings/logs bind mounts stay: config is ro, and Serilog's append-only files
+# have no fsync-durability requirement worth losing `tail -f` from the repo tree for.
+volumes:
+  central-node-a-data:
+    name: scadabridge-central-node-a-data
+  central-node-b-data:
+    name: scadabridge-central-node-b-data
+  site-a-node-a-data:
+    name: scadabridge-site-a-node-a-data
+  site-a-node-b-data:
+    name: scadabridge-site-a-node-b-data
+  site-b-node-a-data:
+    name: scadabridge-site-b-node-a-data
+  site-b-node-b-data:
+    name: scadabridge-site-b-node-b-data
+  site-c-node-a-data:
+    name: scadabridge-site-c-node-a-data
+  site-c-node-b-data:
+    name: scadabridge-site-c-node-b-data
@@ -10,8 +10,8 @@ docker compose -f "$SCRIPT_DIR/docker-compose.yml" down

 echo ""
 echo "Teardown complete."
-echo "Site data (SQLite DBs) and logs are preserved in node directories."
+echo "Node data (SQLite DBs) is preserved in named docker volumes; logs in node directories."
 echo ""
 echo "To also remove persistent data:"
-echo "  rm -rf $SCRIPT_DIR/site-a-node-a/data $SCRIPT_DIR/site-a-node-b/data"
+echo "  docker volume rm \$(docker volume ls -q --filter name=scadabridge-central-node --filter name=scadabridge-site-)"
 echo "  rm -rf $SCRIPT_DIR/*/logs"