50a5291b64
DEV: ~48 s site / ~29 s central outage; QA: pair stop/start, ~2m40s site / ~1m37s central. Chain schema + chainer role gone, env key removed, all gates pass on both tiers. #116 QA site drills (all reactors idle): kill → takeover +7.7 s; stop-and-hold → [leader-auto-down] DOWNING +27.2 s, takeover +27.4 s, recovery restored byte-identical. Claude-Session: https://claude.ai/code/session_01GT67AFKC45MBz1wPZnVbX8
77 lines
6.0 KiB
Markdown
77 lines
6.0 KiB
Markdown
# Deploy pack: ScadaBridge#143, hash-chain removal (`8e32f353` → `061432c8`), DEV vd03 then QA vq03/vq04
|
||
|
||
**Owner go (2026-11-15):** "yes 143, with the 116 dependent on idle". Deploy current `main` to DEV, then QA. The #116 QA site-pair drill follows QA **only if** every QA CvdReactor instance is idle.
|
||
|
||
## Build
|
||
`main` @ `061432c8`. Host and CLI are published with `-c Release -r win-x64 --self-contained true -p:PublishSingleFile=false` (the boxes have no .NET 10). The app has 1056 files and the cli 224. The pins are `ZB.MOM.WW.LocalDb 0.3.8` and `ZB.MOM.WW.Theme 1.0.1`.
|
||
|
||
## Delta `8e32f353..061432c8` (25 commits)
|
||
- **#143 / #107:** the AuditLog hash chain, head export and two-login maintenance hardening are removed (`ed2ed0a3`, `b418408c`, merge `3dea146c`). The archive is kept at format v2. A retired `AuditLog:HashChain*` / `MaintenanceConnectionString` key is ignored and logs one warning.
|
||
- **#141:** the DCL fails over within seconds when the connected OPC UA server is going away (`c4ec51e8`).
|
||
- **#105:** the legacy site-psk-invalidation topic is removed (`89d2becb`).
|
||
- **UI consistency:** Theme 1.0.0 → 1.0.1 (dark palette, reflow), dialog naming and focus containment, controls and editor errors, semantic tables, the audit-window validation and the dashboard count reads (`cdd3adf6`…`6489c9fc`).
|
||
- **Docs and deploy records only:** #52/#142, #116 drills, #107/#138 records.
|
||
- **No appsettings change.** The only option change is a removal (`AuditLog:HashChain`).
|
||
|
||
## Migration 64 → 65
|
||
- `migrate-64-to-65.idempotent.sql` was generated at `061432c8` with `dotnet ef migrations script 20261110120000_AuditMaintenanceGrants --idempotent`. Its SHA-256 is `a16a150de1e1f6c4cb9153e1c53bed95a029d656f18c3dc3b61ec6575198f7a7`, and with comments stripped it is identical to `docs/plans/sql/RemoveAuditLogHashChain.sql`.
|
||
- It applies one migration, `20261115221558_RemoveAuditLogHashChain`, in one transaction, and every statement is existence-guarded.
|
||
- It is run with `SET XACT_ABORT ON` and `sqlcmd -I -b`. On DEV it runs on vd03; on QA it runs on vq03 against wonder-sql-vq03 (vq04 has no `sqlcmd`). The login is the Central service's own connection string, with the password only in `SQLCMDPASSWORD`.
|
||
- **Gate:** `8e32f353` maps `ChainSeq` / `RowHash`, so it cannot run on schema 65. The migration therefore runs with every ScadaBridge service on the tier stopped, and only the new binaries start afterwards.
|
||
|
||
## Env change (Central service only)
|
||
Remove `AuditLog__HashChain__Enabled` and any other `AuditLog__HashChain__*` / `AuditLog__MaintenanceConnectionString` entries. Every other entry must stay byte-identical, and this is proven by a digest (name + SHA-256 prefix, values never printed). On QA the per-node `JwtExpiryMinutes` / `IdleTimeoutMinutes` drift sits in appsettings and is kept.
|
||
|
||
## Order per tier
|
||
1. **Preflight**, read-only:
|
||
- live Host is `8e32f353`;
|
||
- 64 migrations;
|
||
- the chain objects are present: 2 columns, 2 tables, 2 indexes, the `_chainer` role;
|
||
- the env holds `AuditLog__HashChain__Enabled=true`;
|
||
- health and the active node.
|
||
2. **Backup:** `COPY_ONLY, CHECKSUM, COMPRESSION` + `RESTORE VERIFYONLY`; `reg export` of the service keys; appsettings kept with hashes.
|
||
3. **Stop:** DEV stops Site, then Central. QA stops both Sites, then both Centrals, so the pair goes down together.
|
||
4. **Migrate** 64 → 65.
|
||
5. **Swap:** `app` / `cli` → `*.bak.<ts>`, copy in the staged build, restore the kept appsettings (hash match). Re-assert `failureflag 1`.
|
||
6. **Env:** remove the retired keys from the Central env.
|
||
7. **Start:** Central(s) first, then Site(s). On QA vq03 goes first, then vq04.
|
||
|
||
## Gates
|
||
- `/health/ready` 200 on every service, with one active node per pair.
|
||
- ProductVersion `+061432c8` on Host and CLI.
|
||
- 65 migrations.
|
||
- `ChainSeq` / `RowHash`, `AuditLogChainHead` / `AuditLogChainTombstone`, `IX_AuditLog_ChainSeq` / `IX_AuditLog_Unchained` and `scadabridge_audit_chainer` are **absent**; `scadabridge_audit_purger` and `_writer` are present.
|
||
- New AuditLog rows are written after the start.
|
||
- No `no longer supported` (retired-key) warning in the Central log.
|
||
- A bogus inbound key gets 401.
|
||
- Data connections are at baseline. On QA: LocalDb sync connected, Z28062–Z28069 at 244 Good, WarmCanary connected.
|
||
- OTLP `service.version` is `+061432c8`.
|
||
- Overview is Up.
|
||
- The CLI `audit` group has no `verify-chain` / `verify-heads`.
|
||
|
||
## Rollback
|
||
Schema 65 cannot run under `8e32f353`, so a rollback is binaries **plus** a DB restore:
|
||
1. Stop all services.
|
||
2. `RESTORE … WITH REPLACE, CHECKSUM` from the pre-window backup.
|
||
3. `app` / `cli` ← `*.bak.<ts>`.
|
||
4. `reg import` the pre-window Central key, which puts `AuditLog__HashChain__Enabled=true` back.
|
||
5. Start the services: on QA the pair starts together.
|
||
|
||
Never roll back past `0e540291`. The swap scripts auto-roll-back if the start fails. Once the migration has run, that auto-rollback restores the DB.
|
||
|
||
## #116 site-pair drill (QA, conditional)
|
||
Two snapshots about 60 s apart are taken on every QA CvdReactor instance. If any instance has a cycle running, a cycle ID, FurnaceTemp ≥ 60 or a `RecipeDownloadFlag` True, the drill is **skipped**.
|
||
|
||
If every instance is idle:
|
||
1. A kill drill of the active `ScadaBridge-Site` process, picked by service PID.
|
||
2. After about 2 min stable, a stop-and-hold drill: disable the SCM restart, kill, hold ≥ 60 s, capture `[leader-auto-down] DOWNING`, restore the recovery config byte-identical, then start.
|
||
|
||
The gates afterwards are: instances re-hosted at 244 Good, one active site, LocalDb sync connected, and auto-restart restored.
|
||
|
||
## Scripts (as run)
|
||
- `dev-deploy.ps1`: DEV, in one call.
|
||
- `qa.tmpl.ps1`: QA, rendered per step with `__ACTION__` / `__TS__`. The actions are `prep`, `backup`, `stop-site`, `stop-central`, `migrate`, `swap`, `start-central`, `start-site`, `verify` and `rollback-swap`.
|
||
- `drill.tmpl.ps1`: the #116 site drill helper, with actions `status`, `kill`, `disable`, `kill-held`, `restore` and `start`.
|
||
|
||
Results are in `deployment-records/2026-11-15-wonder-app-vd03-061432c8.md` and `…-vq03-vq04-061432c8.md`.
|