From 5b9cf772db6b94b82b02144e584a237d3f2d52e6 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Thu, 20 Aug 2026 16:31:57 -0400 Subject: [PATCH] =?UTF-8?q?docs(deployment):=20record=20QA=20upgrade=20of?= =?UTF-8?q?=20wonder-app-vq03=20+=20vq04=20(all=204=20nodes)=20to=20a24fa3?= =?UTF-8?q?8e=20=E2=80=94=206=20EF=20migrations,=20AuditLog:SiteWriter=20p?= =?UTF-8?q?ath=20added,=20System32=20auditlog.db=20relocated;=20grouped=20?= =?UTF-8?q?bindings=20verified=20on=20Z28061=20(264=20rows=20/=208=20group?= =?UTF-8?q?s)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../2026-08-20-wonder-app-vq03-vq04.md | 61 +++++++++++++++++++ 1 file changed, 61 insertions(+) create mode 100644 docs/deployment/deployment-records/2026-08-20-wonder-app-vq03-vq04.md diff --git a/docs/deployment/deployment-records/2026-08-20-wonder-app-vq03-vq04.md b/docs/deployment/deployment-records/2026-08-20-wonder-app-vq03-vq04.md new file mode 100644 index 00000000..daf1066a --- /dev/null +++ b/docs/deployment/deployment-records/2026-08-20-wonder-app-vq03-vq04.md @@ -0,0 +1,61 @@ +# Deployment record — wonder-app-vq03 + wonder-app-vq04 (QA) — 2026-08-20 + +Point-in-time record of a ScadaBridge binary + schema upgrade of the **QA two-host +topology**: a real central pair (`central-a` = wonder-app-vq03 `10.220.158.2`, `central-b` = +wonder-app-vq04 `10.220.158.3`, :8085) and a site-a pair (`node-a`/`node-b` on the same two +boxes, gRPC :8083, metrics :8084), each box running `ScadaBridge-Central`, `ScadaBridge-Site` +and `ScadaBridge-LDAP` out of `E:\ApiInstall\ScadaBridge`. Shared ConfigDb on +`wonder-sql-vq03`. Performed from the Mac via **psbridge** on vq04 (`:6150`, root `E:\psrem`); +vq03 reached from vq04 over WinRM (`Invoke-Command`) and the `E$` admin share. + +## 1. Binary + schema upgrade + +| Item | Value | +|---|---| +| Build deployed | `main` @ **`a24fa38e`** (from `d4bb4e17`, 2026-08-12; 246 commits) — all four nodes | +| Headline change | **Grouped connection bindings** (`docs/plans/2026-08-20-grouped-connection-bindings-design.md`): the Configure page sources its binding rows from the flattened configuration, grouped per module with a per-group Object apply; CLI `instance bind-module`. Plus everything since 08-12 (arch-review Medium/Low waves, LocalDb 0.3.2→0.3.4, gRPC stream batching, bulk-deploy up-to-date skip, …). | +| Publish | `dotnet publish -c Release -r win-x64 --self-contained true` for Host (→ `app\`, 520 files) and CLI (→ `cli\`), zipped and pushed via `psbridge push` (SHA-256 verified), unpacked under `E:\psrem\releases\a24fa38e\`, mirrored to `\\wonder-app-vq03\E$\psrem-stage\a24fa38e` with robocopy. | +| EF migrations applied | 6, via `sqlcmd -C -b -I` on vq03 from an **idempotent delta script** (`dotnet ef migrations script 20260810100836_AddSmtpConfigurationTransport --idempotent`): `20260814235335_AddAuditLogAndNotificationCoveringIndexes`, `20260815004957_AlignAuditLogEventIdUniqueness`, `20260815154803_AddBundleUnlockAttemptTable`, `20260815165452_AddDeploymentRecordListSortIndex`, `20260819183326_AddSiteCallsListSortIndex`, `20260819201210_AddNotificationsListSortIndex`. All additive (indexes + one table + one index drop). | +| New required config | **`AuditLog:SiteWriter:DatabasePath`** (top-level section, Site nodes; `StartupValidator` refuses to boot without it since `2e4e41a8`) — added to `appsettings.Site.json` on both boxes as `E:\ApiInstall\ScadaBridge\site\data\auditlog.db`. The only appsettings delta across the 246 commits. | +| Data relocated | The site audit SQLite file had been written CWD-relative, i.e. to **`C:\Windows\System32\auditlog.db`** (the exact hazard `2e4e41a8` closed). Moved, with its `-wal`/`-shm`, into `site\data\` on both boxes while the Site service was stopped. | +| Preserved across swap | box `appsettings.json` / `appsettings.Central.json` / `appsettings.Site.json` (copied back from the backup; service env vars untouched), `central\data`, `site\data` (secrets.db, site-localdb.db). | +| DB backup | `ScadaBridge_pre-a24fa38e_20260820T162535.bak` (COPY_ONLY, SQL Server default backup dir on wonder-sql-vq03) | +| Binary backup | `E:\ApiInstall\ScadaBridge\app.bak.20260820T162659` + `cli.bak.20260820T162659` on each box (old `d4bb4e17` binaries + appsettings) | + +### Sequence +Stop Site pair (both boxes) → stop Central pair → swap `app\`/`cli\` on vq04 then vq03 → +start Central pair → wait 20 s → start Site pair. Pairs were stopped and started **together** +(required since LocalDb Phase 2 — no rolling upgrade of a site pair). + +### Pitfalls hit (for next time) +- A **full-history** `--idempotent` script fails on SQL Server batch-compile (`Invalid column + name 'Configuration'` from an early migration) — script **from the DB's current migration**. +- `sqlcmd` needs **`-I`** (QUOTED_IDENTIFIER ON) or the filtered/covering index migrations fail + with Msg 1934. +- Inside a `psbridge exec` single-quoted script, doubled `''` quotes are eaten — use `[char]39`. +- `Invoke-Command` output must be piped through `Out-String` to come back through psbridge. + +### Verification +- `/health/ready` = **200 Healthy** on both centrals (database + akka-cluster Up, memberCount 2, + unreachable 0, all 5 required singletons) and both sites (akka-cluster Up, memberCount 2, + `localdb` Healthy with `replicationConnected=true`, backlog 0). +- Zero `[ERR]`/`[WRN]`/`[FTL]` on any of the four node logs after the new boot (the `[FTL] + host terminated unexpectedly` at 16:27:39 is the *old* build's stop). +- **Feature proven on the QA data that motivated it:** `/deployment/instances/1/configure` + (Z28061, CvdReactor) now lists **264 data-sourced attributes in 8 groups** — `(instance)` 16, + `DelmiaReceiver` 10, `LeftDelmiaReceiver` 10, `LeftMESReceiver` 26, `LeftReactorSide` 83 + (incl. `LeakTest`), `RightDelmiaReceiver` 10, `RightMESReceiver` 26, `RightReactorSide` 83 — + all bound, every group's Object inferred and pre-filled (`Z28061`, `DelmiaReceiver_043`, + `Left_002.DelmiaReceiver`, `MESReceiver_023`, `Left_002`, `Right_002.DelmiaReceiver`, + `MESReceiver_022`, `Right_002`), `Test Bindings (264)`. Before this build the page showed + only the 16 direct attributes. + +## 2. Rollback +- Binaries: on each box stop both services, rename `app` → `app.failed`, `app.bak.20260820T162659` + → `app` (same for `cli`), start Central then Site. The old build tolerates the new (additive) + schema and ignores the added `AuditLog` key — but note it will again write `auditlog.db` + CWD-relative into System32. +- Schema: restore `ScadaBridge_pre-a24fa38e_20260820T162535.bak` (not needed for a binary rollback). + +## 3. Outstanding +- None from this deploy. ZIPSEN (id 7) remains the known non-deployable draft instance.