dohertj2/ScadaBridge
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity

deploy(qa): b3021e7c to vq03/vq04 (55->57, artifacts deployed); SimpleAlarmStatusRequest copied from vd03; register row 126 closed

Browse Source
This commit is contained in:
Joseph Doherty
2026-09-28 11:12:05 -04:00
parent 5cc09b08ef
commit 2aede140e2
2 changed files with 77 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/deployment/deployment-records/2026-09-28-wonder-app-vq03-vq04-b3021e7c.md
+76
View File
@@ -0,0 +1,76 @@
# Deployment record — wonder-app-vq03 + wonder-app-vq04 (QA) — 2026-09-28 — `2b9a7c3c` → `b3021e7c`
Point-in-time record of a ScadaBridge binary + schema upgrade of the **QA two-host topology**
(central pair `central-a` = vq03 `10.220.158.2` / `central-b` = vq04 `10.220.158.3` on :8085, site-a
pair `node-a`/`node-b` on the same boxes, gRPC :8083, health :8084, LocalDb replication ON; services
`ScadaBridge-Central` / `ScadaBridge-Site` out of `E:\ApiInstall\ScadaBridge`; shared ConfigDb
`ScadaBridge` on `wonder-sql-vq03`). Same package as DEV (`2026-09-28-wonder-app-vd03-fe89363b.md`,
second window) — read that for the build delta. Performed from the Mac with the `winrm` CLI
(deployment-only scope: zip over SMB, one prep script per box, seven window calls), after the user
confirmed boxes + build. Closes register row 126.
## 1. Binary + schema upgrade
| Item | Value |
|---|---|
| Build deployed | `main` @ **`b3021e7c`** (from `2b9a7c3c`, 2026-09-10) — all four nodes. Host + CLI ProductVersion `1.0.0+b3021e7c5ff68ba20abc510f644045c7dbbccd86` on both boxes. |
| Headline change | Computed-alarm `IsMesReportable` flag, site artifact staleness + *Pending updates* indicator, Gitea #36 reset-aware handshake, and the read-only allow-listed script `Files` global. |
| Artifact | `E:\psrem-stage\b3021e7c\scadabridge.zip` on both boxes, 125 585 217 B, SHA-256 `5B7D7C68…2D66DF` (re-verified on box; identical to vd03's); unpacked to `…\unpacked` (app 1029 files, cli 223). |
| EF migrations applied | 2 (55 → 57) via `sb-mig-55to57.sql` (from→to, NOT idempotent, identical to vd03's) with `sqlcmd -C -b -I` on vq03, after asserting 55 / `20260909203046_…`: `20260911135501_AddAlarmMesReportable`, `20260914173147_AddSiteArtifactRevision`. Both additive `AddColumn`s — no data upgrade; a binary-only rollback to `2b9a7c3c` would tolerate the extra columns, but the DB restore is the clean path. |
| New required config | **None.** The three `appsettings*.json` per box were kept and hash-matched after the swap (vq03 `F94000B1`/`D1320FA8`/`4CBAF66C`; vq04 `F94000B1`/`2FD3EC4C`/`D0C1AF05` — vq03's Central file changed since 2026-09-10, user-held, restored as-is). Service registry env untouched (`reg export` of both keys to `E:\psrem-stage\b3021e7c\_envbak-*.reg` on each box). **Script file access left OFF** on QA (no `SiteRuntime:ScriptFileAccess:AllowedRoots`). |
| DB backup | `F:\MSSQL\BACKUP\ScadaBridge_pre-b3021e7c_20260928T110651.bak` on wonder-sql-vq03 (COPY_ONLY, CHECKSUM, COMPRESSION; `RESTORE VERIFYONLY … WITH CHECKSUM` valid), taken 11:06:55 with everything running. |
| Binary backup | `app.bak.20260928T110907` / `cli.bak.…` on vq03, `app.bak.20260928T110841` / `cli.bak.…` on vq04 (old `2b9a7c3c` binaries); `_appsettings-keep-<ts>` alongside. |
| Notes on box | `E:\ApiInstall\ScadaBridge\_DEPLOY-NOTES-b3021e7c.txt` on each box. |
### Sequence (maintenance window)
| Step | Local time |
|---|---|
| Stop Site vq04 (**window opens**) | 11:07:34 → 11:08:04 |
| Stop Site vq03 | 11:08:10 → 11:08:21 |
| Stop Central vq04 + swap vq04 | 11:08:28 → 11:08:47 |
| Stop Central vq03, migrate 55 → 57, swap vq03 | 11:08:54 → 11:09:13 |
| Start Central vq03 (was active) → ready | 11:09:13 → 11:09:27 |
| Start Central vq04 → ready | 11:09:35 → 11:09:43 |
| Start Site vq03 → ready | 11:09:52 → 11:10:00 |
| Start Site vq04 → ready (**window closes**) | 11:10:08 → 11:10:12 |
**Downtime (first stop → last site ready): 2 min 38 s.** Pairs stopped and started together.
### Verification (from the Mac, HTTP + CLI)
- `:8085 /health/ready` 200 on both centrals — database Healthy, akka-cluster memberCount 2 /
unreachable 0, required-singletons Healthy; `/health/active` 200 vq03 / 503 vq04 (same active
node as before). Ready also proves the production-mode pending-migration check passed (57/57).
- `:8084 /health/ready` 200 on both sites — memberCount 2, `localdb` `replicationConnected:true`;
`/health/active` 200 vq03 / 503 vq04.
- `POST /management` no-auth → 401 (unchanged).
- Snapshots: Z28061 / Z28062 244 attrs, 244 Good before (11:07) and after (11:11).
- **Deploy Artifacts** (last step, per row 126): deployment `999dee70…`, site-a success;
`Sites.ArtifactRevisionHash` = `sha256:d79ba1a3…`, `ArtifactsDeployedAt` 15:11:20Z — Sites page
reads *Current*, not *Never deployed*.
## 2. Inbound method copied from DEV
`SimpleAlarmStatusRequest` (QA id 9) was a stub returning `WasSuccessful=true, Alarms=[]`. Replaced
via `api-method update` with vd03's definition (id 9): script (BTDB `dbo.Machine` SAPID → code,
trailing A/B stripped → `Route.To(instance).Call("AlarmStatus", FlaggedOnly/IncludeAcked)`),
return definition (now `required: [WasSuccessful, Alarms]`), parameters `{SAPID}`, timeout 30 s —
field-for-field identical to vd03 afterwards. Key-less smoke via `flow-debug run-method`:
`SAPID=000000` → `Failed to find machine with SAPID '000000'` (BTDB query ran, 113 ms);
`SAPID=""` → same shape. **No QA instance carries an `AlarmStatus` script yet**, so a real SAPID
answers `Alarm status is not supported on machine '<code>'` until the MES templates on QA gain it.
## 3. Rollback
1. Stop Site vq04 → vq03, Central vq04 → vq03; confirm no `ZB.MOM.WW.ScadaBridge.Host` process.
2. From vq03: `RESTORE DATABASE [ScadaBridge] FROM DISK = 'F:\MSSQL\BACKUP\ScadaBridge_pre-b3021e7c_20260928T110651.bak' WITH REPLACE`
(reverts the two columns and the method-9 update).
3. Each box: `app` → `app.failed`, `app.bak.<ts>` → `app`; same for `cli`.
4. Start Central vq03 → vq04, then Site vq03 → vq04.
## 4. Outstanding
- `ApiKeyStore:Provider` still unset on the QA pair (per-node SQLite inbound keys) — unchanged.
- Housekeeping after soak: the 2026-09-10 and this deploy's `app.bak.*`/`cli.bak.*`/`_appsettings-keep-*`
and `E:\psrem-stage\{2b9a7c3c,b3021e7c}` on both boxes.
docs/plans/2026-07-08-deferred-work-register.md
+1 -1
View File
@@ -119,7 +119,7 @@ All 7 fix-now items landed via PLAN-04/05/06/07/08 (verified in review 08 round
| 123 | **Rail nav scroll pane (`.rail-scroll` + the `#theme-rail > .side-rail { height: 100% }` fill rule) lives in app-local `site.css`/`NavMenu.razor`, not the `ZB.MOM.WW.Theme` kit** *(Low)*. Every other kit consumer (OtOpcUa, MxGateway, HistorianGateway, ignitionoee) has the same unbounded-nav-pushes-the-footer-off-screen shape and would need the identical fix copy-pasted. | `docs/plans/2026-09-14-pending-updates-design.md` §3.5 | Scoped to ScadaBridge for this change; upstreaming into the shared kit is a separate cross-repo release, not a one-app CSS tweak. | A second kit consumer hits the same long-nav clipping and copies the fix instead of consuming it from the kit. |
| 124 | **No template-graph-watermark short-circuit on the pending-updates fleet flatten** *(Low)*. `PendingUpdatesEvaluator.EvaluateAsync` re-flattens every deployed instance through `DeploymentService.GetInstanceStalenessAsync` on every cache-TTL expiry (`PendingUpdatesCacheTtl`, default 60 s) while any Deployer/Admin circuit is open, even when nothing in the template/instance/artifact graph has changed since the last evaluation. | `docs/plans/2026-09-14-pending-updates-design.md` §3.2 ("Not in scope") | The single-flight TTL cache already bounds the cost to one fleet flatten per TTL fleet-wide (not per viewer); a watermark short-circuit is a further optimization, not a correctness need, and needs a cheap "has anything changed" signal that does not exist yet. | The fleet flatten's cost (instance count x template depth) becomes measurable against `PendingUpdatesCacheTtl` on a large deployment. |
| 125 | **Pending-updates counts have no Health-dashboard tile or CLI surface** *(Low)*. The aggregate (`PendingUpdatesSnapshot.StaleInstanceCount` / `StaleArtifactSiteCount`) is consumed only by the Central UI Sites page and rail indicator; there is no Health dashboard KPI tile and no `deploy`/`site` CLI command surfacing it — `site list` shows only the raw `artifactRevisionHash`/`artifactsDeployedAt` fields on each site, not whether they are current. | `docs/plans/2026-09-14-pending-updates-design.md` §3.2 ("Not in scope") | Out of scope for the UI-focused ask; the CLI/Health surfaces are a separate consumer of the same `IPendingUpdatesCache`. | An operator wants the pending-updates counts without opening the Central UI (headless monitoring, CLI scripting). |
| 126 | **TODO — roll the two outstanding migrations to the QA pair vq03/vq04** *(deployment, open)*. **DEV vd03 DONE 2026-09-28** (`fe89363b`, 55 → 57, record `docs/deployment/deployment-records/2026-09-28-wonder-app-vd03-fe89363b.md` — the `sb-mig-55to57.sql` there is the same script QA needs). QA still runs `2b9a7c3c` with the DB at migration 55 (`20260909203046_EnforceGlobalInstanceNamesAndProtectProtocolConfiguration`); `main` (`ac833f5a`, pushed 2026-09-14) carries two more: **56** `20260911135501_AddAlarmMesReportable` (computed-alarm `IsMesReportable` flag) and **57** `20260914173147_AddSiteArtifactRevision` (`Sites.ArtifactRevisionHash`/`ArtifactsDeployedAt`, both nullable). Both are additive `AddColumn`s — no data upgrade, no coordinated step like the 2026-09-10 protocol-config encryption. Recipe (same as the 55 rollout): generate ONE from→to delta script `dotnet ef migrations script EnforceGlobalInstanceNamesAndProtectProtocolConfiguration AddSiteArtifactRevision --project src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase --startup-project src/ZB.MOM.WW.ScadaBridge.Host -o sb-mig-55to57.sql` (NOT idempotent, identical for vd03 and the QA pair), take the pre-upgrade DB backup, stop the pair TOGETHER, apply via `sqlcmd -C -b -I` on the SQL host, publish/swap the `ac833f5a` binaries, start the pair, confirm the Central boot log shows 0 pending migrations and no "Applying migration" (the script already applied them). After upgrade every site reads *Artifacts never deployed* until one *Deploy Artifacts to All Sites* — run it as the last step. Write the deployment record `docs/deployment/deployment-records/<date>-wonder-app-{vd03,vq03-vq04}-ac833f5a.md`. | Pending-updates finish (2026-09-14) | ~~Blocked by the 2026-09-11 remote-access hold~~ — **unblocked 2026-09-28**: `winrm` is re-enabled for deployment activities (CLAUDE.md). vd03 done; QA waits only for the user to schedule its window. | The user asks for the QA (vq03/vq04) window. |
| 126 | ~~**TODO — roll the two outstanding migrations to the QA pair vq03/vq04**~~ **DONE 2026-09-28** — QA vq03/vq04 upgraded `2b9a7c3c` → `b3021e7c`, 55 → 57, artifacts deployed (record `docs/deployment/deployment-records/2026-09-28-wonder-app-vq03-vq04-b3021e7c.md`). *(deployment, closed)*. **DEV vd03 DONE 2026-09-28** (`fe89363b`, 55 → 57, record `docs/deployment/deployment-records/2026-09-28-wonder-app-vd03-fe89363b.md` — the `sb-mig-55to57.sql` there is the same script QA needs). QA still runs `2b9a7c3c` with the DB at migration 55 (`20260909203046_EnforceGlobalInstanceNamesAndProtectProtocolConfiguration`); `main` (`ac833f5a`, pushed 2026-09-14) carries two more: **56** `20260911135501_AddAlarmMesReportable` (computed-alarm `IsMesReportable` flag) and **57** `20260914173147_AddSiteArtifactRevision` (`Sites.ArtifactRevisionHash`/`ArtifactsDeployedAt`, both nullable). Both are additive `AddColumn`s — no data upgrade, no coordinated step like the 2026-09-10 protocol-config encryption. Recipe (same as the 55 rollout): generate ONE from→to delta script `dotnet ef migrations script EnforceGlobalInstanceNamesAndProtectProtocolConfiguration AddSiteArtifactRevision --project src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase --startup-project src/ZB.MOM.WW.ScadaBridge.Host -o sb-mig-55to57.sql` (NOT idempotent, identical for vd03 and the QA pair), take the pre-upgrade DB backup, stop the pair TOGETHER, apply via `sqlcmd -C -b -I` on the SQL host, publish/swap the `ac833f5a` binaries, start the pair, confirm the Central boot log shows 0 pending migrations and no "Applying migration" (the script already applied them). After upgrade every site reads *Artifacts never deployed* until one *Deploy Artifacts to All Sites* — run it as the last step. Write the deployment record `docs/deployment/deployment-records/<date>-wonder-app-{vd03,vq03-vq04}-ac833f5a.md`. | Pending-updates finish (2026-09-14) | ~~Blocked by the 2026-09-11 remote-access hold~~ — **unblocked 2026-09-28**: `winrm` is re-enabled for deployment activities (CLAUDE.md). vd03 done; QA done 2026-09-28. | Closed 2026-09-28. |
## Resolved (verified against the code 2026-07-10)
Rows removed from the Deferred table above once confirmed shipped. Kept here for traceability.