docs(deploy): record DEV wonder-app-vd03 upgrade b3021e7c -> 3808b49f (migration 58, gate 31/31)

docs/deployment/deployment-records/2026-10-22-wonder-app-vd03-3808b49f.md

@@ -0,0 +1,84 @@
+# 2026-10-22 — DEV (wonder-app-vd03) `b3021e7c` → `3808b49f`
+
+**Build:** `main` @ `3808b49f` (= `origin/main`), ProductVersion `1.0.0+3808b49fc7558a177219d00eac97747b28850a80`,
+published from a detached worktree at that SHA (`dotnet publish` Host + CLI `-c Release -r win-x64 --self-contained true
+-p:PublishSingleFile=false`; 1028 app files, 223 cli files, zip 125 543 156 B, SHA-256 `770bb992…5fb1098e47`).
+**Delta:** 191 commits. Highlights:
+- site secrets ride Deploy Artifacts (the gRPC secrets hub is retired);
+- the 2026-10-21 system-review remediation (CLI bearer tokens, config-audit retention, actor-name gate, per-lane S&F sweep,
+  audit forwarding states, …);
+- Topology fixes and features: area rename persists (#38), first-layer preload (#39), **Deploy all…** plus CLI `deploy area` (#40).
+
+**QA (vq03/vq04) and prod NOT touched.**
+
+**Transport:** the same as 2026-09-28.
+- The Mac-side `winrm` CLI ran two on-box scripts (`sb-prep.ps1`, `sb-swap.ps1`) over WinRM 5985.
+- Files went over SMB 445.
+- Baseline, verification and logs went over HTTP and SMB, with DB reads as a direct SQL login.
+
+## Pre-flight (breaking-change check, done before the window)
+
+| Area | Change in delta | vd03 state | Verdict |
+|---|---|---|---|
+| Schema | 57 → 58: `ReviewRemediationIndexes`, which adds 3 `AuditLogEntries` indexes and 2 partition-aligned `AuditLog` indexes | 57; AuditLog 1 670 rows, AuditLogEntries 465; `ps_AuditLog_Month` present | Additive, seconds to build |
+| `Secrets:Replication:Enabled` ≠ false | Now a hard `StartupValidator` failure | `false` in appsettings.json; not in the service env | OK |
+| `ScadaBridge:Security:JwtSigningKey` | Must be ≥ 32 bytes when set | Env value, 64 chars | OK |
+| `ScadaBridge:Database:MachineDataDb` | Option removed (it was a required key) | Still in the Central service env and appsettings | Ignored, harmless; can be removed |
+| `Transport:SchemaVersionMajor`, `JwtRefreshThresholdMinutes` | Options removed | Not set | OK |
+| Site deploy actor-name gate (`ActorNameRules`) | Rejects names with spaces or non-ASCII | Checked all 37 instance, script, alarm, native-source and connection names | All valid |
+| Data-connection `secret:` references | Artifact deploy now resolves them centrally and fails closed | The MxGateway connection carries no reference; the artifact hash is unchanged after the deploy | OK |
+| CLI auth | The new CLI exchanges credentials at `/auth/token` | The new server answers 404 under DisableLogin, so the CLI falls back to Basic | OK (verified) |
+| Package changes | `Secrets.Replicator.Grpc` dropped; crypto/SignalR pins raised | n/a | OK |
+
+**No appsettings change.** The box's three `appsettings*.json` were kept and restored, with SHA-256 matches:
+- appsettings.json `A6B6CF05…`
+- appsettings.Central.json `3382DD83…`
+- appsettings.Site.json `3E7AD9A4…`
+
+The service registry environment was untouched.
+
+## Procedure (`E:\psrem-stage\3808b49f\`, log `E:\ApiInstall\ScadaBridge\_DEPLOY-NOTES-20261022T151735.txt`)
+
+1. **Baseline** (CLI `debug snapshot`):
+   - Z28061 (1), Z28062 (2) and Z28061Sim (8): Bad 270 / 270. This was **already all-Bad before the window**: the MxGateway worker had been timing out on `AddItemBulk` all day, about 2 450 such site-log lines before 15:17.
+   - SBTEST01 (9) and SBTEST02 (10): Good 86, 5 alarms each.
+2. **`sb-prep.ps1`** (15:17:08):
+   - Zip hash re-verified and unpacked; staged Host and CLI are `3808b49f`.
+   - `reg export` of both service keys.
+   - `BACKUP DATABASE … COPY_ONLY, CHECKSUM, COMPRESSION` → `E:\ApiInstall\ScadaBridge-backups\ScadaBridge_pre-3808b49f_20261022T151708.bak` (130 453 504 B); `VERIFYONLY` valid.
+3. **`sb-swap.ps1`**:
+   - 15:17:36: Site, then Central stopped (10 s each).
+   - 15:17:57: `ReviewRemediationIndexes.sql` applied with `sqlcmd -b -I`; migrations 57 → 58.
+   - `app`/`cli` renamed to `*.bak.20261022T151735`; staged copy in; appsettings restored.
+   - Central ready at 15:18:16; Site ready at **15:18:24**.
+   - **About 49 s of downtime.**
+   - SQL Server warned that `IX_AuditLog_Kind_Occurred` has a 8 008-byte maximum key (the `Kind` computed column is wide). `Kind` holds short enum names, so this never bites in practice.
+4. **`deploy artifacts --site-id 2`**: success.
+   - `artifactsDeployedAt 2026-10-22T19:20:28Z`.
+   - Hash unchanged (`sha256:9e054ef0…`), as designed for a site with no secret references.
+
+## Verification
+
+| Check | Result |
+|---|---|
+| `:8085` and `:8084` `/health/ready` | 200 / 200 |
+| ProductVersion | Live Host and CLI are `3808b49f` |
+| DB | 58 migrations |
+| `POST /auth/token` | 404 (DisableLogin); the new CLI falls back to Basic in the default token mode |
+| Snapshots after | Identical to the baseline |
+| Logs after boot (pulled over SMB) | **Central:** the usual first-probe `akka-cluster … Removed` ERR lines plus two expected Warnings (below). **Site:** only the pre-existing MxGateway `AddItemBulk` deadline / quality-Bad lines |
+| `docker/verify-mes-test-machine.sh` (with the `RECIPE_ROOT` / `RECIPE_OUTSIDE` for vd03) | **PASS 31/31** |
+
+The two expected Central Warnings both come from the placeholder second seed. That entry makes the single-node central count as clustered:
+- inbound API keys are on SQLite;
+- **new:** secrets use a per-node SQLite store.
+
+## Rollback
+
+1. Stop Site, then Central.
+2. Rename `app` → `app.failed` and `app.bak.20261022T151735` → `app`; do the same for `cli`.
+3. Start Central, then Site.
+
+The new indexes are additive and `b3021e7c` ignores the extra migration row, so no DB restore is needed. To make the DB exact anyway, `RESTORE DATABASE [ScadaBridge] FROM DISK = 'E:\ApiInstall\ScadaBridge-backups\ScadaBridge_pre-3808b49f_20261022T151708.bak' WITH REPLACE`.
+
+Staging is left under `E:\psrem-stage\3808b49f\`. The `_appsettings-keep-20261022T151735` and `*.bak.20261022T151735` folders can go once the build has soaked.