From 9789b01acd9dd7f63ee290dcb785c85ebeddd2c9 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Thu, 19 Nov 2026 16:14:01 -0500 Subject: [PATCH] =?UTF-8?q?deploy(ldap):=20QA=20vq03/vq04=20NAM=20AD=20cut?= =?UTF-8?q?over=20executed=20with=20the=20DEV=20builds=20=E2=80=94=20HG,?= =?UTF-8?q?=20mxgw,=20ScadaBridge,=20OtOpcUa=20(record=20+=20scripts)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Claude-Session: https://claude.ai/code/session_01GT67AD5mc2BnaaAfY3hqqQ --- docs/deploy/2026-11-19-qa-nam-ldap/README.md | 98 +++++++++++++ .../2026-11-19-qa-nam-ldap/env-edit.ps1 | 19 +++ docs/deploy/2026-11-19-qa-nam-ldap/env-hg.ps1 | 35 +++++ .../2026-11-19-qa-nam-ldap/env-mxgw.ps1 | 34 +++++ .../2026-11-19-qa-nam-ldap/env-otopcua.ps1 | 37 +++++ .../mxgw-upgrade.tmpl.ps1 | 24 ++++ .../otopcua-prepare.ps1 | 80 +++++++++++ .../2026-11-19-qa-nam-ldap/otopcua-start.ps1 | 27 ++++ .../2026-11-19-qa-nam-ldap/otopcua-swap.ps1 | 33 +++++ .../2026-11-19-qa-nam-ldap/otopcua-swap2.ps1 | 50 +++++++ .../2026-11-19-qa-nam-ldap/preflight.ps1 | 33 +++++ .../2026-11-19-qa-nam-ldap/sb-qa.tmpl.ps1 | 131 ++++++++++++++++++ .../2026-11-19-qa-nam-ldap/seed-secret.ps1 | 46 ++++++ docs/plans/2026-11-15-nam-ldap-cutover.md | 7 +- 14 files changed, 652 insertions(+), 2 deletions(-) create mode 100644 docs/deploy/2026-11-19-qa-nam-ldap/README.md create mode 100644 docs/deploy/2026-11-19-qa-nam-ldap/env-edit.ps1 create mode 100644 docs/deploy/2026-11-19-qa-nam-ldap/env-hg.ps1 create mode 100644 docs/deploy/2026-11-19-qa-nam-ldap/env-mxgw.ps1 create mode 100644 docs/deploy/2026-11-19-qa-nam-ldap/env-otopcua.ps1 create mode 100644 docs/deploy/2026-11-19-qa-nam-ldap/mxgw-upgrade.tmpl.ps1 create mode 100644 docs/deploy/2026-11-19-qa-nam-ldap/otopcua-prepare.ps1 create mode 100644 docs/deploy/2026-11-19-qa-nam-ldap/otopcua-start.ps1 create mode 100644 docs/deploy/2026-11-19-qa-nam-ldap/otopcua-swap.ps1 create mode 100644 docs/deploy/2026-11-19-qa-nam-ldap/otopcua-swap2.ps1 create mode 100644 docs/deploy/2026-11-19-qa-nam-ldap/preflight.ps1 create mode 100644 docs/deploy/2026-11-19-qa-nam-ldap/sb-qa.tmpl.ps1 create mode 100644 docs/deploy/2026-11-19-qa-nam-ldap/seed-secret.ps1 diff --git a/docs/deploy/2026-11-19-qa-nam-ldap/README.md b/docs/deploy/2026-11-19-qa-nam-ldap/README.md new file mode 100644 index 0000000..4e7eea7 --- /dev/null +++ b/docs/deploy/2026-11-19-qa-nam-ldap/README.md @@ -0,0 +1,98 @@ +# 2026-11-19: QA vq03/vq04 NAM AD LDAP cutover + DEV builds (HistorianGateway, MxAccessGateway, ScadaBridge, OtOpcUa) + +**Owner go (2026-11-19):** "do the qa ldap plan, winrm usage approved". Plan: +[`docs/plans/2026-11-15-nam-ldap-cutover.md`](../../plans/2026-11-15-nam-ldap-cutover.md) §0Q (svc_igntadmin interim, DEV builds, +one window). DEV record: [`../2026-11-19-vd03-nam-ldap/`](../2026-11-19-vd03-nam-ldap/README.md). Umbrella: scadaproj#6. + +**Result: all four apps on both QA boxes now run the DEV builds and sign users in against NAM AD** over LDAPS (nam1, nam2 +as fallback), bound as `svc_igntadmin@nam.zimmer.com` with the password from `${secret:nam-ldap-svc-password}` in each +app's own store. The GLAuth `multi-role` identity no longer signs in anywhere on QA. **Zero service-account bind +failures** and zero LDAP warnings in either box's OTel log export for the whole window. + +| App | Build | Where the LDAP config now lives | +|---|---|---| +| HistorianGateway (per box) | `6365f34` → **`f84af29`** | NSSM env `Ldap__*` + `Dashboard__GroupToRole__*` | +| MxAccessGateway (per box) | `76a71bc8` → **`e74918cc`** | NSSM env `MxGateway__Ldap__*` + `MxGateway__Dashboard__GroupToRole__*`; base `app\appsettings.json` lost `GwAdmin`/`GwReader` | +| ScadaBridge (pair) | `061432c8` → **`2e0ce81e`** | each Central's `app\appsettings.Central.json` (`ScadaBridge:Security:Ldap`); roles in `LdapGroupMappings` | +| OtOpcUa (Warm pair) | `76e73e7d` → **`cdf433a3`** | service env `Security__Ldap__*` (+ `Security__Cookie__RequireHttpsCookie=false`) | + +Every artifact is the one DEV runs, pulled from vd03's stage dirs over SMB and re-hashed (HG zip `1595a828…`, mxgw +`f0999b75…`, OtOpcUa `c89ba972…`, ScadaBridge `6c706b0a…`). No app had a schema change. + +## Steps (EST) + +| Time | Step | +|---|---| +| 15:29 | **Phase 0** (`preflight.ps1`, both boxes): nam1/nam2 reachable on 636 + 389; `zimmer-rootca` in `LocalMachine\Root` (`78D941D7…`, exp 2036); LDAPS chains validate (TLS 1.3, no policy errors). Every service key and `appsettings*.json` → `E:\ApiInstall\_ldapcut-20261119T152947` (vq03) / `…T152958` (vq04), protected DACL. (The first run died on a parsing bug after one `reg export`; its `…T152920`/`…T152930` dirs hold only that file.) OtOpcUa Primary **vq04**, ScadaBridge active central **vq03**. HG and mxgw already had `RequireHttpsCookie=false` in env; OtOpcUa had no cookie setting | +| 15:35 | Staged on both boxes: the four builds under `E:\psrem-stage\{hg-f84af29,mxgw-e74918cc,otopcua-cdf433a3,sb-2e0ce81e}`, the Secrets CLI 0.6.6 at `E:\ApiInstall\_tools\secret-0.6.6\`. Hashes checked on box | +| 15:37 | **ScadaBridge role rows by CLI** as GLAuth `multi-role` against the active central (audited path): ids 6–10, `UG_WW_APP_ADMIN_QA`→Administrator, `UG_WW_APP_IDE_QA`→Designer, `UG_WW_APP_IDE_RO_QA`→Viewer, **`UG_WW_HIST_ADMIN_QA`→Deployer (interim)**, `UG_WW_BTMGR_RO_QA`→Viewer | +| 15:41 | **Secret seeded** (`seed-secret.ps1`; each store's path and KEK read from its own service env): HG, mxgw, OtOpcUa on each box (SQLite, KEK files under `C:\ProgramData\ZB\kek\`), ScadaBridge **once**, into the shared SQL Server store (`zbsecrets` schema; now 1 row). Read-back matched in all 7; the value file was ACL'd and deleted. Both Centrals' `scadabridge.kek` are identical (`A3BD293A…`). See finding 2 for the SQL Server seeding | +| 15:43–15:48 | **HistorianGateway** vq03 then vq04: `env-hg.ps1` (34 → 41 entries: NAM `Ldap__*`, QA `GroupToRole`; `SCADA-*` dropped), then the f84af29 pack (`preflight.ps1` 0 FAIL / 0 WARN incl. a side-by-side boot with the new env, `swap.ps1`, `verify.ps1` 0 FAIL). Downtime **6.6 s / 7.7 s** | +| 15:56–15:59 | **MxAccessGateway** vq03 then vq04: `env-mxgw.ps1` (33 → 38: NAM `MxGateway__Ldap__*`, QA `GroupToRole`; `SCADA-*` dropped), then `mxgw-upgrade.tmpl.ps1` (puts the NAM base `appsettings.json` in place, then the repo upgrade script with the `START_PENDING` try/catch patch from vd03; it carries the live appsettings into the new `app\`). Dry run first on vq03. Env fingerprint unchanged through stop/swap/start/60 s stability on both | +| 16:00 | **ScadaBridge** `prep` both nodes; DB backup `F:\MSSQL\BACKUP\ScadaBridge_pre-2e0ce81e_20261119T1600.bak` (VERIFYONLY ok; 65 migrations) | +| 16:00:53–16:04:14 | ScadaBridge window (`sb-qa.tmpl.ps1`): stop Site vq03, vq04, Central vq03, vq04; swap both (binaries + each node's own NAM `appsettings.Central.json`; `ScadaBridge__Security__Ldap__ServiceAccountPassword` removed, 15 → 14 env entries); start Central vq03 (16:03:17), vq04 (16:03:37), Site vq03 (16:03:55), vq04 (16:04:14). **Site data path ~2 min 17 s, central API ~55 s** | +| 16:05–16:10 | **OtOpcUa** (`otopcua-prepare/swap2/start.ps1`, the 11-13 QA pack retargeted): prepare both (1410 files, deps ok, ConfigDb `OtOpcUaConfig_pre-cdf433a3_20261119.bak` + VERIFYONLY on vq03, 9 migrations, isolation `-Verify` OK); swap the Standby vq03 (`-NoStart`), `env-otopcua.ps1` (51 → 61); swap vq04 (stop 16:08:33), env (50 → 60); start vq04 (ready 16:09:29, 37.8 s), vq03 (ready 16:09:49, 8 s). Full isolation `-Verify` **OK 21/21** on both. **Pair outage ~56 s** | + +## Mappings now on QA + +| Group | OtOpcUa | ScadaBridge | MxAccessGateway | HistorianGateway | +|---|---|---|---|---| +| `UG_WW_APP_ADMIN_QA` | Administrator | Administrator | Administrator | Administrator | +| `UG_WW_APP_IDE_QA` | Designer | Designer | Viewer | — | +| `UG_WW_APP_IDE_RO_QA` | Viewer | Viewer | Viewer | Viewer | +| `UG_WW_HIST_ADMIN_QA` | Viewer | **Deployer (interim)** | — | Administrator | +| `UG_WW_BTMGR_RO_QA` | — | Viewer | — | Viewer | + +ScadaBridge still holds the five `SCADA-*` seed rows (ids 1–5); they grant nothing in NAM. + +## Verification + +| Check | HG `:5220` | mxgw `:5130` | ScadaBridge `:8085` | OtOpcUa `:9000` | +|---|---|---|---|---| +| Anonymous `GET /` → login | 302 both | 302 both | 302 | 302 both | +| Bogus user refused | ✓ both | ✓ both | ✓ (GLAuth `multi-role` refused too, UI + CLI) | ✓ both | +| `dohertj2` signs in | ✓ both | ✓ both | ✓ cookie; `/auth/token` 200 roles **Administrator+Designer+Deployer**; CLI (HTTP Basic) `instance list` (11) + admin `role-mapping list` | ✓ cookie, `GET /` 200 (no redirect loop); `/auth/token` 200 roles **Viewer+Designer+Administrator** | +| After | pack `verify.ps1` 0 FAIL both; `ReadAggregate` gate as expected | `/health/ready` 200, env unchanged | central + site ready on both; active central vq03; site-a both nodes online; **MxGateway Connected 2196/2196 Good**; **WarmCanary Connected (to the Primary, vq04) 2/2 Good** after the OtOpcUa restart; LocalDb replication connected | **vq04 Primary 250 (`warm_role` 1), vq03 Standby 90**; `localdb_oplog_depth` 0 both; script workers up (shards 0, 1) both | + +Overview `:5320` untouched. **Not done, by owner instruction:** DC-failover checks and firewall rules. **Not done:** the +optional oswaltk Viewer-only login (step 8; needs that user). + +## Findings + +1. **HG pack assumed QA was on `fa9cf31`.** QA has run `6365f34` since 11-13. The pack's QA source SHA was patched + (preflight/swap) before the run; nothing else in the pack needed changing. +2. **The headless Secrets CLI cannot write the SQL Server store.** `secret set` registers only `AddZbSecrets` (SQLite); + the interactive console does honour `Secrets:SqlServer`, but needs a TTY. ScadaBridge's Central uses + `Secrets:Store:Provider=SqlServer`, so a one-off CLI build (0.6.6 + "if `Secrets:Store:Provider=SqlServer`, call + `AddZbSecretsSqlServerStore`", mirroring ScadaBridge's own registration) seeded it, staged at + `E:\ApiInstall\_tools\secret-0.6.6-sql\` on vq03. **Follow-up:** make the headless CLI honour the provider key the + same way, in `ZB.MOM.WW.Secrets`. +3. **mxgw on QA keeps its LDAP settings in the NSSM env**, not in `appsettings.json` (unlike vd03). The base file's + `GwAdmin`/`GwReader` merge with the env `GroupToRole`, so they were still removed from the file. The repo upgrade + script carries the live `appsettings*.json` forward, so future redeploys keep the edit. +4. **OtOpcUa's first start after a pair stop took 37.8 s** (vq04, alone in the mesh); the pair outage was ~56 s against + ~37 s on 11-13. The swap itself had no `file in use` retry this time (the #908 fix is in the cdf433a3 installer). +5. Noise seen, not caused here: OtOpcUa "Mesh PSK secret `ot-mesh-psk-dev` for cluster DEV is absent" (QA has no DEV + cluster mesh peer); HG plaintext/relative-path production-readiness warnings. + +## Rollback (per app) + +Keys and files: `E:\ApiInstall\_ldapcut-20261119T152947` (vq03) / `…T152958` (vq04) (`reg-.reg`, `_app_appsettings*.json`). +- **HG:** the pack's `rollback.ps1` (→ `app.bak.20261119T154443` vq03 / `…T154740` vq04), then `reg import reg-HistorianGateway.reg`, restart. +- **mxgw:** `Upgrade-MxGatewayService.ps1 -Rollback` (stamps `20261119-155641` vq03 / `20261119-155819` vq04), then `reg import reg-MxAccessGw.reg` and restore `MxAccessGateway_app_appsettings.json`, restart. +- **ScadaBridge:** `rollback-swap` action (`app.bak.20261119T1600`, Central env from `_envbak-ScadaBridge-Central-20261119T1600.reg`), start Central vq03, vq04, then the Sites. No DB restore needed. Keep the `UG_WW_*` rows. +- **OtOpcUa:** stop both; swap `app.bak.20261119-pre-cdf433a3` back; `reg import reg-OtOpcUaHost.reg` on each; start both. + +The seeded secrets are inert once nothing references them. The box-local GLAuth (`ScadaBridge-LDAP`) is still running +until the soak ends. + +## Automation identity + +QA automation now uses **`NAM\dohertj2`** (HTTP Basic / `--password-file`; Keychain `winrm-cli/vq03`). No checked-in QA +script used `multi-role`; the remaining `multi-role` users are the local Docker rig scripts, which stay on GLAuth. + +## Files + +`preflight.ps1`, `seed-secret.ps1`, `env-edit.ps1` (+ rendered `env-hg.ps1`, `env-mxgw.ps1`, `env-otopcua.ps1`), +`mxgw-upgrade.tmpl.ps1`, `sb-qa.tmpl.ps1`, `otopcua-prepare.ps1`, `otopcua-swap.ps1`, `otopcua-swap2.ps1`, +`otopcua-start.ps1`. Login probes reused `../2026-11-19-vd03-nam-ldap/login-test.sh`, `sb-cli.sh`, `otelgrep.py`. diff --git a/docs/deploy/2026-11-19-qa-nam-ldap/env-edit.ps1 b/docs/deploy/2026-11-19-qa-nam-ldap/env-edit.ps1 new file mode 100644 index 0000000..895b719 --- /dev/null +++ b/docs/deploy/2026-11-19-qa-nam-ldap/env-edit.ps1 @@ -0,0 +1,19 @@ +# Rewrite one service's env block (NSSM AppEnvironmentExtra or SCM Environment): drop entries whose NAME matches +# $Drop, append $Add. Every other entry stays byte-identical (checked). No restart. Rendered per call. +$ErrorActionPreference = 'Stop' +$Svc = '__SVC__'; $Kind = '__KIND__' # nssm | svc +$Drop = '__DROP__' +$Add = @(__ADD__) +$k = "HKLM:\SYSTEM\CurrentControlSet\Services\$Svc" +if ($Kind -eq 'nssm') { $k = "$k\Parameters"; $prop = 'AppEnvironmentExtra' } else { $prop = 'Environment' } +$before = @((Get-ItemProperty $k -Name $prop).$prop) +$kept = @($before | Where-Object { $n = $_.Substring(0, [Math]::Max(0,$_.IndexOf('='))); $n -notmatch $Drop }) +$dropped = @($before | Where-Object { $kept -notcontains $_ } | ForEach-Object { $_.Substring(0,$_.IndexOf('=')) }) +foreach ($a in $Add) { $n = $a.Substring(0,$a.IndexOf('=')); if (@($kept | Where-Object { $_ -like "$n=*" }).Count) { throw "add collides with kept entry $n" } } +$new = $kept + $Add +Set-ItemProperty -Path $k -Name $prop -Value ([string[]]$new) -Type MultiString +$after = @((Get-ItemProperty $k -Name $prop).$prop) +if ($after.Count -ne $new.Count) { throw 'count mismatch after write' } +for ($i = 0; $i -lt $new.Count; $i++) { if ($after[$i] -cne $new[$i]) { throw "entry $i differs after write" } } +"$Svc ($prop) $($before.Count) -> $($after.Count) entries; dropped: $($dropped -join ', ')" +foreach ($a in $Add) { " + $a" } diff --git a/docs/deploy/2026-11-19-qa-nam-ldap/env-hg.ps1 b/docs/deploy/2026-11-19-qa-nam-ldap/env-hg.ps1 new file mode 100644 index 0000000..4c8db12 --- /dev/null +++ b/docs/deploy/2026-11-19-qa-nam-ldap/env-hg.ps1 @@ -0,0 +1,35 @@ +# Rewrite one service's env block (NSSM AppEnvironmentExtra or SCM Environment): drop entries whose NAME matches +# $Drop, append $Add. Every other entry stays byte-identical (checked). No restart. Rendered per call. +$ErrorActionPreference = 'Stop' +$Svc = 'HistorianGateway'; $Kind = 'nssm' # nssm | svc +$Drop = '^(Ldap__|Dashboard__GroupToRole__)' +$Add = @('Ldap__Enabled=true', + 'Ldap__Server=uswap-vdc-nam1.nam.zimmer.com', + 'Ldap__Port=636', + 'Ldap__Transport=Ldaps', + 'Ldap__AllowInsecure=false', + 'Ldap__FallbackServers__0=uswap-vdc-nam2.nam.zimmer.com:636', + 'Ldap__SearchBase=DC=nam,DC=zimmer,DC=com', + 'Ldap__ServiceAccountDn=svc_igntadmin@nam.zimmer.com', + 'Ldap__ServiceAccountPassword=${secret:nam-ldap-svc-password}', + 'Ldap__UserNameAttribute=sAMAccountName', + 'Ldap__DisplayNameAttribute=name', + 'Ldap__GroupAttribute=memberOf', + 'Ldap__BindMode=ServiceAccount', + 'Dashboard__GroupToRole__UG_WW_APP_ADMIN_QA=Administrator', + 'Dashboard__GroupToRole__UG_WW_APP_IDE_RO_QA=Viewer', + 'Dashboard__GroupToRole__UG_WW_HIST_ADMIN_QA=Administrator', + 'Dashboard__GroupToRole__UG_WW_BTMGR_RO_QA=Viewer') +$k = "HKLM:\SYSTEM\CurrentControlSet\Services\$Svc" +if ($Kind -eq 'nssm') { $k = "$k\Parameters"; $prop = 'AppEnvironmentExtra' } else { $prop = 'Environment' } +$before = @((Get-ItemProperty $k -Name $prop).$prop) +$kept = @($before | Where-Object { $n = $_.Substring(0, [Math]::Max(0,$_.IndexOf('='))); $n -notmatch $Drop }) +$dropped = @($before | Where-Object { $kept -notcontains $_ } | ForEach-Object { $_.Substring(0,$_.IndexOf('=')) }) +foreach ($a in $Add) { $n = $a.Substring(0,$a.IndexOf('=')); if (@($kept | Where-Object { $_ -like "$n=*" }).Count) { throw "add collides with kept entry $n" } } +$new = $kept + $Add +Set-ItemProperty -Path $k -Name $prop -Value ([string[]]$new) -Type MultiString +$after = @((Get-ItemProperty $k -Name $prop).$prop) +if ($after.Count -ne $new.Count) { throw 'count mismatch after write' } +for ($i = 0; $i -lt $new.Count; $i++) { if ($after[$i] -cne $new[$i]) { throw "entry $i differs after write" } } +"$Svc ($prop) $($before.Count) -> $($after.Count) entries; dropped: $($dropped -join ', ')" +foreach ($a in $Add) { " + $a" } diff --git a/docs/deploy/2026-11-19-qa-nam-ldap/env-mxgw.ps1 b/docs/deploy/2026-11-19-qa-nam-ldap/env-mxgw.ps1 new file mode 100644 index 0000000..4106d3f --- /dev/null +++ b/docs/deploy/2026-11-19-qa-nam-ldap/env-mxgw.ps1 @@ -0,0 +1,34 @@ +# Rewrite one service's env block (NSSM AppEnvironmentExtra or SCM Environment): drop entries whose NAME matches +# $Drop, append $Add. Every other entry stays byte-identical (checked). No restart. Rendered per call. +$ErrorActionPreference = 'Stop' +$Svc = 'MxAccessGw'; $Kind = 'nssm' # nssm | svc +$Drop = '^(MxGateway__Ldap__|MxGateway__Dashboard__GroupToRole__)' +$Add = @('MxGateway__Ldap__Enabled=true', + 'MxGateway__Ldap__Server=uswap-vdc-nam1.nam.zimmer.com', + 'MxGateway__Ldap__Port=636', + 'MxGateway__Ldap__Transport=Ldaps', + 'MxGateway__Ldap__AllowInsecure=false', + 'MxGateway__Ldap__FallbackServers__0=uswap-vdc-nam2.nam.zimmer.com:636', + 'MxGateway__Ldap__SearchBase=DC=nam,DC=zimmer,DC=com', + 'MxGateway__Ldap__ServiceAccountDn=svc_igntadmin@nam.zimmer.com', + 'MxGateway__Ldap__ServiceAccountPassword=${secret:nam-ldap-svc-password}', + 'MxGateway__Ldap__UserNameAttribute=sAMAccountName', + 'MxGateway__Ldap__DisplayNameAttribute=name', + 'MxGateway__Ldap__GroupAttribute=memberOf', + 'MxGateway__Ldap__BindMode=ServiceAccount', + 'MxGateway__Dashboard__GroupToRole__UG_WW_APP_ADMIN_QA=Administrator', + 'MxGateway__Dashboard__GroupToRole__UG_WW_APP_IDE_QA=Viewer', + 'MxGateway__Dashboard__GroupToRole__UG_WW_APP_IDE_RO_QA=Viewer') +$k = "HKLM:\SYSTEM\CurrentControlSet\Services\$Svc" +if ($Kind -eq 'nssm') { $k = "$k\Parameters"; $prop = 'AppEnvironmentExtra' } else { $prop = 'Environment' } +$before = @((Get-ItemProperty $k -Name $prop).$prop) +$kept = @($before | Where-Object { $n = $_.Substring(0, [Math]::Max(0,$_.IndexOf('='))); $n -notmatch $Drop }) +$dropped = @($before | Where-Object { $kept -notcontains $_ } | ForEach-Object { $_.Substring(0,$_.IndexOf('=')) }) +foreach ($a in $Add) { $n = $a.Substring(0,$a.IndexOf('=')); if (@($kept | Where-Object { $_ -like "$n=*" }).Count) { throw "add collides with kept entry $n" } } +$new = $kept + $Add +Set-ItemProperty -Path $k -Name $prop -Value ([string[]]$new) -Type MultiString +$after = @((Get-ItemProperty $k -Name $prop).$prop) +if ($after.Count -ne $new.Count) { throw 'count mismatch after write' } +for ($i = 0; $i -lt $new.Count; $i++) { if ($after[$i] -cne $new[$i]) { throw "entry $i differs after write" } } +"$Svc ($prop) $($before.Count) -> $($after.Count) entries; dropped: $($dropped -join ', ')" +foreach ($a in $Add) { " + $a" } diff --git a/docs/deploy/2026-11-19-qa-nam-ldap/env-otopcua.ps1 b/docs/deploy/2026-11-19-qa-nam-ldap/env-otopcua.ps1 new file mode 100644 index 0000000..470dd66 --- /dev/null +++ b/docs/deploy/2026-11-19-qa-nam-ldap/env-otopcua.ps1 @@ -0,0 +1,37 @@ +# Rewrite one service's env block (NSSM AppEnvironmentExtra or SCM Environment): drop entries whose NAME matches +# $Drop, append $Add. Every other entry stays byte-identical (checked). No restart. Rendered per call. +$ErrorActionPreference = 'Stop' +$Svc = 'OtOpcUaHost'; $Kind = 'svc' # nssm | svc +$Drop = '^(Security__Ldap__|Security__Cookie__RequireHttpsCookie$)' +$Add = @('Security__Ldap__DevStubMode=false', + 'Security__Ldap__Enabled=true', + 'Security__Ldap__Server=uswap-vdc-nam1.nam.zimmer.com', + 'Security__Ldap__Port=636', + 'Security__Ldap__Transport=Ldaps', + 'Security__Ldap__AllowInsecure=false', + 'Security__Ldap__FallbackServers__0=uswap-vdc-nam2.nam.zimmer.com:636', + 'Security__Ldap__SearchBase=DC=nam,DC=zimmer,DC=com', + 'Security__Ldap__ServiceAccountDn=svc_igntadmin@nam.zimmer.com', + 'Security__Ldap__ServiceAccountPassword=${secret:nam-ldap-svc-password}', + 'Security__Ldap__UserNameAttribute=sAMAccountName', + 'Security__Ldap__DisplayNameAttribute=name', + 'Security__Ldap__GroupAttribute=memberOf', + 'Security__Ldap__BindMode=ServiceAccount', + 'Security__Ldap__GroupToRole__UG_WW_APP_ADMIN_QA=Administrator', + 'Security__Ldap__GroupToRole__UG_WW_APP_IDE_QA=Designer', + 'Security__Ldap__GroupToRole__UG_WW_APP_IDE_RO_QA=Viewer', + 'Security__Ldap__GroupToRole__UG_WW_HIST_ADMIN_QA=Viewer', + 'Security__Cookie__RequireHttpsCookie=false') +$k = "HKLM:\SYSTEM\CurrentControlSet\Services\$Svc" +if ($Kind -eq 'nssm') { $k = "$k\Parameters"; $prop = 'AppEnvironmentExtra' } else { $prop = 'Environment' } +$before = @((Get-ItemProperty $k -Name $prop).$prop) +$kept = @($before | Where-Object { $n = $_.Substring(0, [Math]::Max(0,$_.IndexOf('='))); $n -notmatch $Drop }) +$dropped = @($before | Where-Object { $kept -notcontains $_ } | ForEach-Object { $_.Substring(0,$_.IndexOf('=')) }) +foreach ($a in $Add) { $n = $a.Substring(0,$a.IndexOf('=')); if (@($kept | Where-Object { $_ -like "$n=*" }).Count) { throw "add collides with kept entry $n" } } +$new = $kept + $Add +Set-ItemProperty -Path $k -Name $prop -Value ([string[]]$new) -Type MultiString +$after = @((Get-ItemProperty $k -Name $prop).$prop) +if ($after.Count -ne $new.Count) { throw 'count mismatch after write' } +for ($i = 0; $i -lt $new.Count; $i++) { if ($after[$i] -cne $new[$i]) { throw "entry $i differs after write" } } +"$Svc ($prop) $($before.Count) -> $($after.Count) entries; dropped: $($dropped -join ', ')" +foreach ($a in $Add) { " + $a" } diff --git a/docs/deploy/2026-11-19-qa-nam-ldap/mxgw-upgrade.tmpl.ps1 b/docs/deploy/2026-11-19-qa-nam-ldap/mxgw-upgrade.tmpl.ps1 new file mode 100644 index 0000000..dc09e61 --- /dev/null +++ b/docs/deploy/2026-11-19-qa-nam-ldap/mxgw-upgrade.tmpl.ps1 @@ -0,0 +1,24 @@ +# MxAccessGateway QA 76a71bc8 -> e74918cc with the NAM base appsettings.json. Rendered: __MODE__ = dry | real. +# The repo upgrade script carries the live appsettings*.json into the new app\, so the NAM base file is put in place +# first (GwAdmin/GwReader removed; the LDAP settings themselves live in the NSSM env, written by env-mxgw.ps1). +$ErrorActionPreference = 'Stop' +$Mode = '__MODE__' +$Stage = 'E:\psrem-stage\mxgw-e74918cc' +$Zip = Join-Path $Stage 'mxgw-e74918cc.zip' +$want = 'F0999B75BEE196375AEB8E3E277A0E948C54A747FD5251D336461C9665D847F2' +$got = (Get-FileHash $Zip -Algorithm SHA256).Hash +if ($got -ne $want) { "ABORT zip hash $got"; exit 1 } +$exp = Join-Path $Stage 'expanded' +if (-not (Test-Path (Join-Path $exp 'Worker\MxGateway.Worker.exe'))) { Expand-Archive -Path $Zip -DestinationPath $exp -Force } +"staged Server = " + [Diagnostics.FileVersionInfo]::GetVersionInfo((Join-Path $exp 'Server\ZB.MOM.WW.MxGateway.Server.dll')).ProductVersion +"staged Worker = " + [Diagnostics.FileVersionInfo]::GetVersionInfo((Join-Path $exp 'Worker\MxGateway.Worker.exe')).ProductVersion +$live = 'E:\ApiInstall\MxAccessGateway\app\appsettings.json' +if ($Mode -eq 'real') { + $nam = Join-Path $Stage 'appsettings.json.nam' + Get-Content $nam -Raw | ConvertFrom-Json | Out-Null + Copy-Item $nam $live -Force + "live appsettings.json <- NAM base, sha256 $((Get-FileHash $live).Hash.Substring(0,16))" +} +$a = @{ ServerSource = (Join-Path $exp 'Server'); WorkerSource = (Join-Path $exp 'Worker'); NssmPath = 'C:\ProgramData\ZB\qa-build\tools\nssm.exe'; ExpectedSha = 'e74918cc'; Label = 'nam-e74918cc' } +if ($Mode -ne 'real') { $a.DryRun = $true } +& (Join-Path $Stage 'Upgrade-MxGatewayService.ps1') @a 2>&1 | Out-String -Width 300 diff --git a/docs/deploy/2026-11-19-qa-nam-ldap/otopcua-prepare.ps1 b/docs/deploy/2026-11-19-qa-nam-ldap/otopcua-prepare.ps1 new file mode 100644 index 0000000..8712ce9 --- /dev/null +++ b/docs/deploy/2026-11-19-qa-nam-ldap/otopcua-prepare.ps1 @@ -0,0 +1,80 @@ +# QA OtOpcUa upgrade c9640f5c -> cdf433a3, phase 1 on ONE node (service keeps running): protected backup dir, registry +# export, script + zip hashes, expand to app.new-cdf433a3 + deps check, env NAMES/SHA prefixes (+ no OpcUa__Redundancy* / +# PeerApplicationUris*), pre-window -Verify with the new installer, swap -WhatIf. On vq03 only (-DbBackup): ConfigDb +# COPY_ONLY+CHECKSUM+COMPRESSION backup + RESTORE VERIFYONLY + migration state. Secrets never printed. +param([switch]$DbBackup) +$ErrorActionPreference = 'Stop' +try { +$stageDir = 'E:\psrem-stage\otopcua-cdf433a3' +$backupDir = "$stageDir\backup" +$candidate = 'E:\ApiInstall\OtOpcUa\app.new-cdf433a3' +$svcKeyPath = 'HKLM:\SYSTEM\CurrentControlSet\Services\OtOpcUaHost' +. "$stageDir\envdigest.ps1" +Write-Output "node=$env:COMPUTERNAME" +if (-not (Test-Path -LiteralPath $backupDir)) { New-Item -ItemType Directory -Force -Path $backupDir | Out-Null } +& icacls.exe $backupDir /inheritance:r /grant:r 'SYSTEM:(OI)(CI)(F)' 'BUILTIN\Administrators:(OI)(CI)(F)' | Out-Null +if ($LASTEXITCODE -ne 0) { throw "icacls backup dir failed $LASTEXITCODE" } +Write-Output "backup dir $backupDir (protected: SYSTEM + Administrators)" +& reg.exe export 'HKLM\SYSTEM\CurrentControlSet\Services\OtOpcUaHost' "$backupDir\OtOpcUaHost-pre-cdf433a3.reg" /y | Out-Null +if ($LASTEXITCODE -ne 0) { throw "reg export failed $LASTEXITCODE" } +Write-Output "reg export ok" +$expectScripts = @{ 'Install-ScriptWorkerIsolation.ps1' = 'BB31D2759D0DE29292C123D74860226ED37AF554F2C8FE6871312C7E81F7C05A'; + 'ScriptWorkerIsolation.psm1' = '4C1C9D0B7086D66C3977E013933018D7495C14DB56E53701633688B10AB9D32C'; + 'Invoke-OtOpcUaInstallSwap.ps1' = '10C07BD0F527BDBDACC2B94605FB410996F0B9446390F88A8DF5C725AB0C7C6F' } +foreach ($scriptName in $expectScripts.Keys) { + if ((Get-FileHash -LiteralPath "$stageDir\install\$scriptName" -Algorithm SHA256).Hash -ne $expectScripts[$scriptName]) { throw "script hash mismatch $scriptName" } +} +Write-Output "install scripts: 3/3 hashes match the pack" +$wantHash = ((Get-Content -LiteralPath "$stageDir\otopcua-cdf433a3.zip.sha256" -Raw).Trim() -split '\s+')[0] +$gotHash = (Get-FileHash -LiteralPath "$stageDir\otopcua-cdf433a3.zip" -Algorithm SHA256).Hash +if ($wantHash -ne $gotHash) { throw "zip hash mismatch want=$wantHash got=$gotHash" } +if (Test-Path -LiteralPath $candidate) { Remove-Item -LiteralPath $candidate -Recurse -Force } +Expand-Archive -LiteralPath "$stageDir\otopcua-cdf433a3.zip" -DestinationPath $candidate +$fileCount = (Get-ChildItem -LiteralPath $candidate -Recurse -File | Measure-Object).Count +Write-Output ("expanded files={0} (expect 1410) zip hash ok {1}" -f $fileCount, $gotHash.Substring(0,12)) +if ($fileCount -ne 1410) { throw "file count mismatch" } +$verInfo = (Get-Item -LiteralPath "$candidate\OtOpcUa.Host.dll").VersionInfo.ProductVersion +Write-Output "candidate ProductVersion=$verInfo" +if ($verInfo -notlike '1.0.0+cdf433a3512353de*') { throw "unexpected ProductVersion" } +$depsText = Get-Content -LiteralPath "$candidate\OtOpcUa.Host.deps.json" -Raw +foreach ($pkg in 'ZB.MOM.WW.LocalDb/0.3.8','ZB.MOM.WW.LocalDb.Replication/0.3.8','ZB.MOM.WW.HistorianGateway.Client/1.1.0') { + if ($depsText -notmatch [regex]::Escape("`"$pkg`"")) { throw "deps.json lacks $pkg" } +} +Write-Output "deps.json: LocalDb + .Replication 0.3.8, HistorianGateway.Client 1.1.0" +$envLines = @((Get-ItemProperty $svcKeyPath).Environment) +Show-ZbEnvDigest $envLines 'pre' +$envTable = @{} +foreach ($envLine in $envLines) { $eqPos = $envLine.IndexOf('='); if ($eqPos -gt 0) { $envTable[$envLine.Substring(0,$eqPos)] = $envLine.Substring($eqPos+1) } } +foreach ($flagName in 'Scripting__Runner','Scripting__Runner__Seams__0','Scripting__Runner__Seams__1','Scripting__Runner__Seams__2','Scripting__Worker__Identity','Scripting__Worker__PeCacheDir','OtOpcUa__Telemetry__Exporter','OtOpcUa__Telemetry__OtlpEndpoint') { + if ($envTable.ContainsKey($flagName)) { Write-Output " flag $flagName=$($envTable[$flagName])" } else { Write-Output " flag $flagName=" } +} +$redundancyNames = @($envTable.Keys | Where-Object { $_ -like 'OpcUa__Redundancy*' -or $_ -like 'OpcUa__PeerApplicationUris*' }) +Write-Output (" OpcUa__Redundancy*/PeerApplicationUris* entries: {0}" -f $redundancyNames.Count) +# QA runs Warm since 2026-11-13: redundancy keys are expected (recorded above, not refused) +$appsettingsHits = @(Get-ChildItem -LiteralPath 'E:\ApiInstall\OtOpcUa\app' -Filter 'appsettings*.json' | Select-String -Pattern 'Redundancy|PeerApplicationUris') +Write-Output (" appsettings*.json Redundancy/PeerApplicationUris hits: {0}" -f $appsettingsHits.Count) + +if ($DbBackup) { + $connBuilder = New-Object System.Data.SqlClient.SqlConnectionStringBuilder($envTable['ConnectionStrings__ConfigDb']) + $sqlcmdExe = (Get-Command sqlcmd.exe -ErrorAction SilentlyContinue).Source + if (-not $sqlcmdExe) { $sqlcmdExe = 'C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\170\Tools\Binn\SQLCMD.EXE' } + $env:SQLCMDPASSWORD = $connBuilder.Password + try { + & $sqlcmdExe -S $connBuilder.DataSource -U $connBuilder.UserID -C -I -b -d master -W -h -1 -Q "BACKUP DATABASE [OtOpcUaConfig] TO DISK = N'OtOpcUaConfig_pre-cdf433a3_20261119.bak' WITH COPY_ONLY, INIT, CHECKSUM, COMPRESSION; RESTORE VERIFYONLY FROM DISK = N'OtOpcUaConfig_pre-cdf433a3_20261119.bak' WITH CHECKSUM; SELECT 'db backup + verify ok';" + if ($LASTEXITCODE -ne 0) { throw "db backup failed $LASTEXITCODE" } + & $sqlcmdExe -S $connBuilder.DataSource -U $connBuilder.UserID -C -I -b -d $connBuilder.InitialCatalog -W -h -1 -Q "SET NOCOUNT ON; SELECT 'migrations=' + CAST(COUNT(*) AS varchar(10)) + ' last=' + MAX(MigrationId) FROM __EFMigrationsHistory; SELECT 'snapshot=' + CAST(snapshot_isolation_state_desc AS varchar(20)) FROM sys.databases WHERE name = DB_NAME(); SELECT TOP 1 'latest sealed=' + LOWER(CONVERT(varchar(36),DeploymentId)) + ' rev=' + RevisionHash FROM dbo.Deployment WHERE Status = 2 ORDER BY SealedAtUtc DESC;" + if ($LASTEXITCODE -ne 0) { throw "migration query failed $LASTEXITCODE" } + } finally { Remove-Item Env:SQLCMDPASSWORD -ErrorAction SilentlyContinue; Remove-Variable connBuilder -ErrorAction SilentlyContinue } +} +Set-Location "$stageDir\install" +Write-Output '--- pre-window -Verify (new installer, service up)' +& powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\Install-ScriptWorkerIsolation.ps1 -Verify *>&1 | ForEach-Object { "$_" } | Where-Object { $_ -notmatch 'does not inherit the worker deny|grants nothing to Everyone|^\s*(@\{|-DenyPaths\.|to -DenyPaths\.|add it to -DenyPaths\.|RESTRICTED, or add)' } +$verifyExit = $LASTEXITCODE +Write-Output "pre-window verify exit=$verifyExit" +if ($verifyExit -ne 0) { throw "pre-window verify failed" } +Write-Output '--- swap -WhatIf (preflight only)' +& .\Invoke-OtOpcUaInstallSwap.ps1 -InstallRoot E:\ApiInstall\OtOpcUa\app -Candidate $candidate -PreviousInstall E:\ApiInstall\OtOpcUa\app.bak.20261119-pre-cdf433a3 -Carry 'appsettings*.json','logs' -WhatIf *>&1 | ForEach-Object { "$_" } +Write-Output ("E: free {0} GB" -f [int]((Get-PSDrive E).Free/1GB)) +Write-Output "service=$((Get-Service OtOpcUaHost).Status)" +exit 0 +} catch { Write-Output "PREPARE FAILED: $($_.Exception.Message)"; exit 1 } diff --git a/docs/deploy/2026-11-19-qa-nam-ldap/otopcua-start.ps1 b/docs/deploy/2026-11-19-qa-nam-ldap/otopcua-start.ps1 new file mode 100644 index 0000000..b27973e --- /dev/null +++ b/docs/deploy/2026-11-19-qa-nam-ldap/otopcua-start.ps1 @@ -0,0 +1,27 @@ +# QA: start OtOpcUaHost on ONE node, wait for /health/ready 200 (up to 180 s), full -Verify, env digest, live version. +$ErrorActionPreference = 'Stop' +function Out-ZbStamped { process { '{0:HH:mm:ss.fff}Z {1}' -f (Get-Date).ToUniversalTime(), "$_" } } +try { +$stageDir = 'E:\psrem-stage\otopcua-cdf433a3' +. "$stageDir\envdigest.ps1" +"node=$env:COMPUTERNAME" | Out-ZbStamped +$t0 = Get-Date +'start issued' | Out-ZbStamped +Start-Service OtOpcUaHost +$deadline = (Get-Date).AddSeconds(180) +$readyCode = 0 +while ((Get-Date) -lt $deadline) { + if ((Get-Service OtOpcUaHost).Status -ne 'Running') { Start-Sleep -Milliseconds 500; continue } + try { $readyCode = (Invoke-WebRequest -UseBasicParsing -Uri 'http://localhost:9000/health/ready' -TimeoutSec 5).StatusCode } catch { $readyCode = 0 } + if ($readyCode -eq 200) { break } + Start-Sleep -Milliseconds 500 +} +("service={0} ready={1} ({2:N1}s after start issued)" -f (Get-Service OtOpcUaHost).Status, $readyCode, ((Get-Date)-$t0).TotalSeconds) | Out-ZbStamped +Set-Location "$stageDir\install" +& powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\Install-ScriptWorkerIsolation.ps1 -Verify *>&1 | ForEach-Object { "$_" } | Where-Object { $_ -notmatch 'does not inherit the worker deny|grants nothing to Everyone|^\s*(@\{|-DenyPaths\.|to -DenyPaths\.|add it to -DenyPaths\.|RESTRICTED, or add)' } | Where-Object { $_ -match 'VERIFY|MISSING|coverage' } +"post-start verify exit=$LASTEXITCODE" | Out-ZbStamped +Show-ZbEnvDigest @((Get-ItemProperty 'HKLM:\SYSTEM\CurrentControlSet\Services\OtOpcUaHost').Environment) 'post' | Select-Object -First 1 +Write-Output ("live ProductVersion=" + (Get-Item 'E:\ApiInstall\OtOpcUa\app\OtOpcUa.Host.dll').VersionInfo.ProductVersion) +if ($readyCode -ne 200) { exit 1 } +exit 0 +} catch { Write-Output "START FAILED: $($_.Exception.Message)"; exit 1 } diff --git a/docs/deploy/2026-11-19-qa-nam-ldap/otopcua-swap.ps1 b/docs/deploy/2026-11-19-qa-nam-ldap/otopcua-swap.ps1 new file mode 100644 index 0000000..b041c80 --- /dev/null +++ b/docs/deploy/2026-11-19-qa-nam-ldap/otopcua-swap.ps1 @@ -0,0 +1,33 @@ +# QA window on ONE node, started with the service RUNNING (pack §4 QA steps 2/3): Invoke-OtOpcUaInstallSwap.ps1 +# -Carry appsettings*.json,logs -NoStart runs -Scope Outside before its own stop, stops, swaps, -Scope InstallRoot +# reapply+verify, leaves it STOPPED. Then back up data\ + pki\ (stopped => consistent SQLite triplets). Lines UTC-stamped. +$ErrorActionPreference = 'Stop' +function Out-ZbStamped { process { '{0:HH:mm:ss.fff}Z {1}' -f (Get-Date).ToUniversalTime(), "$_" } } +try { +$stageDir = 'E:\psrem-stage\otopcua-cdf433a3' +$backupDir = "$stageDir\backup" +$candidate = 'E:\ApiInstall\OtOpcUa\app.new-cdf433a3' +$prevInstall = 'E:\ApiInstall\OtOpcUa\app.bak.20261119-pre-cdf433a3' +"node=$env:COMPUTERNAME" | Out-ZbStamped +if (-not (Test-Path -LiteralPath $candidate)) { throw "candidate missing" } +if (Test-Path -LiteralPath $prevInstall) { throw "prev path already exists: $prevInstall" } +if ((Get-Service OtOpcUaHost).Status -ne 'Running') { throw "service not running; the pack starts the swap with the service up" } +'swap invoked (service running)' | Out-ZbStamped +Set-Location "$stageDir\install" +& .\Invoke-OtOpcUaInstallSwap.ps1 -InstallRoot E:\ApiInstall\OtOpcUa\app -Candidate $candidate -PreviousInstall $prevInstall ` + -Carry 'appsettings*.json','logs' -NoStart *>&1 | ForEach-Object { "$_" } | + Where-Object { $_ -notmatch 'does not inherit the worker deny|grants nothing to Everyone|^\s*(@\{|-DenyPaths\.|to -DenyPaths\.|add it to -DenyPaths\.|RESTRICTED, or add)' } | Out-ZbStamped +("swap returned, service={0}" -f (Get-Service OtOpcUaHost).Status) | Out-ZbStamped +if ((Get-Service OtOpcUaHost).Status -ne 'Stopped') { throw "service not stopped after the -NoStart swap" } +if (-not (Test-Path -LiteralPath $prevInstall)) { throw "swap did not move the live folder" } +if (Get-Process -Name 'OtOpcUa.Host' -ErrorAction SilentlyContinue) { throw "OtOpcUa.Host still running" } +$workerProc = Get-Process -Name 'ZB.MOM.WW.OtOpcUa.ScriptWorker' -ErrorAction SilentlyContinue +if ($workerProc) { Start-Sleep 3; if (Get-Process -Name 'ZB.MOM.WW.OtOpcUa.ScriptWorker' -ErrorAction SilentlyContinue) { throw "ScriptWorker still running" } } +foreach ($dirName in 'data','pki') { + & robocopy.exe "E:\ApiInstall\OtOpcUa\$dirName" "$backupDir\$dirName-pre-cdf433a3" /E /COPY:DAT /R:1 /W:1 /NFL /NDL /NJH /NP | Out-Null + if ($LASTEXITCODE -ge 8) { throw "$dirName backup robocopy failed $LASTEXITCODE" } +} +"data + pki backup ok -> $backupDir" | Out-ZbStamped +("staged ProductVersion=" + (Get-Item 'E:\ApiInstall\OtOpcUa\app\OtOpcUa.Host.dll').VersionInfo.ProductVersion) | Out-ZbStamped +exit 0 +} catch { Write-Output "SWAP FAILED: $($_.Exception.Message)"; Write-Output "service=$((Get-Service OtOpcUaHost).Status)"; exit 1 } diff --git a/docs/deploy/2026-11-19-qa-nam-ldap/otopcua-swap2.ps1 b/docs/deploy/2026-11-19-qa-nam-ldap/otopcua-swap2.ps1 new file mode 100644 index 0000000..efa2ef7 --- /dev/null +++ b/docs/deploy/2026-11-19-qa-nam-ldap/otopcua-swap2.ps1 @@ -0,0 +1,50 @@ +# QA window on ONE node, started with the service RUNNING (pack §4 QA steps 2/3): Invoke-OtOpcUaInstallSwap.ps1 +# -Carry appsettings*.json,logs -NoStart runs -Scope Outside before its own stop, stops, swaps, -Scope InstallRoot +# reapply+verify, leaves it STOPPED. Then back up data\ + pki\ (stopped => consistent SQLite triplets). Lines UTC-stamped. +$ErrorActionPreference = 'Stop' +function Out-ZbStamped { process { '{0:HH:mm:ss.fff}Z {1}' -f (Get-Date).ToUniversalTime(), "$_" } } +try { +$stageDir = 'E:\psrem-stage\otopcua-cdf433a3' +$backupDir = "$stageDir\backup" +$candidate = 'E:\ApiInstall\OtOpcUa\app.new-cdf433a3' +$prevInstall = 'E:\ApiInstall\OtOpcUa\app.bak.20261119-pre-cdf433a3' +"node=$env:COMPUTERNAME" | Out-ZbStamped +if (-not (Test-Path -LiteralPath $candidate)) { throw "candidate missing" } +if (Test-Path -LiteralPath $prevInstall) { throw "prev path already exists: $prevInstall" } +if ((Get-Service OtOpcUaHost).Status -ne 'Running') { throw "service not running; the pack starts the swap with the service up" } +'swap invoked (service running)' | Out-ZbStamped +Set-Location "$stageDir\install" +$swapFilter = 'does not inherit the worker deny|grants nothing to Everyone|^\s*(@\{|-DenyPaths\.|to -DenyPaths\.|add it to -DenyPaths\.|RESTRICTED, or add)' +try { + & .\Invoke-OtOpcUaInstallSwap.ps1 -InstallRoot E:\ApiInstall\OtOpcUa\app -Candidate $candidate -PreviousInstall $prevInstall ` + -Carry 'appsettings*.json','logs' -NoStart *>&1 | ForEach-Object { "$_" } | Where-Object { $_ -notmatch $swapFilter } | Out-ZbStamped +} catch { + # vq04 2026-11-13: the move right after the stop hit "file in use" (host/worker still exiting after SCM said Stopped). + # Retry once only when nothing moved: service Stopped, app\ intact, candidate still staged, no PreviousInstall. + ("first swap attempt failed: {0}" -f $_.Exception.Message) | Out-ZbStamped + if ((Get-Service OtOpcUaHost).Status -ne 'Stopped' -or (Test-Path -LiteralPath $prevInstall) -or -not (Test-Path -LiteralPath $candidate) -or + -not (Test-Path -LiteralPath 'E:\ApiInstall\OtOpcUa\app\OtOpcUa.Host.dll')) { throw "state not retryable: $($_.Exception.Message)" } + $waitUntil = (Get-Date).AddSeconds(30) + while ((Get-Date) -lt $waitUntil -and (Get-Process -Name 'OtOpcUa.Host','ZB.MOM.WW.OtOpcUa.ScriptWorker' -ErrorAction SilentlyContinue)) { Start-Sleep -Milliseconds 500 } + $left = @(Get-Process -Name 'OtOpcUa.Host','ZB.MOM.WW.OtOpcUa.ScriptWorker' -ErrorAction SilentlyContinue) + ("host/worker processes left: {0}" -f (($left | ForEach-Object { "$($_.ProcessName)#$($_.Id)" }) -join ', ')) | Out-ZbStamped + foreach ($p in $left) { Stop-Process -Id $p.Id -Force } + Start-Sleep -Seconds 2 + 'retrying the swap (service stopped)' | Out-ZbStamped + & .\Invoke-OtOpcUaInstallSwap.ps1 -InstallRoot E:\ApiInstall\OtOpcUa\app -Candidate $candidate -PreviousInstall $prevInstall ` + -Carry 'appsettings*.json','logs' -NoStart *>&1 | ForEach-Object { "$_" } | Where-Object { $_ -notmatch $swapFilter } | Out-ZbStamped +} +("swap returned, service={0}" -f (Get-Service OtOpcUaHost).Status) | Out-ZbStamped +if ((Get-Service OtOpcUaHost).Status -ne 'Stopped') { throw "service not stopped after the -NoStart swap" } +if (-not (Test-Path -LiteralPath $prevInstall)) { throw "swap did not move the live folder" } +if (Get-Process -Name 'OtOpcUa.Host' -ErrorAction SilentlyContinue) { throw "OtOpcUa.Host still running" } +$workerProc = Get-Process -Name 'ZB.MOM.WW.OtOpcUa.ScriptWorker' -ErrorAction SilentlyContinue +if ($workerProc) { Start-Sleep 3; if (Get-Process -Name 'ZB.MOM.WW.OtOpcUa.ScriptWorker' -ErrorAction SilentlyContinue) { throw "ScriptWorker still running" } } +foreach ($dirName in 'data','pki') { + & robocopy.exe "E:\ApiInstall\OtOpcUa\$dirName" "$backupDir\$dirName-pre-cdf433a3" /E /COPY:DAT /R:1 /W:1 /NFL /NDL /NJH /NP | Out-Null + if ($LASTEXITCODE -ge 8) { throw "$dirName backup robocopy failed $LASTEXITCODE" } +} +"data + pki backup ok -> $backupDir" | Out-ZbStamped +("staged ProductVersion=" + (Get-Item 'E:\ApiInstall\OtOpcUa\app\OtOpcUa.Host.dll').VersionInfo.ProductVersion) | Out-ZbStamped +exit 0 +} catch { Write-Output "SWAP FAILED: $($_.Exception.Message)"; Write-Output "service=$((Get-Service OtOpcUaHost).Status)"; exit 1 } diff --git a/docs/deploy/2026-11-19-qa-nam-ldap/preflight.ps1 b/docs/deploy/2026-11-19-qa-nam-ldap/preflight.ps1 new file mode 100644 index 0000000..6b70b3c --- /dev/null +++ b/docs/deploy/2026-11-19-qa-nam-ldap/preflight.ps1 @@ -0,0 +1,33 @@ +# NAM LDAP cutover, QA vq03/vq04 — Phase 0 preflight + backups (read-only except the backup folder). Secret values are never printed. +$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue' +$ts = Get-Date -Format 'yyyyMMddTHHmmss' +$bk = "E:\ApiInstall\_ldapcut-$ts"; New-Item -ItemType Directory -Force $bk | Out-Null +& icacls.exe $bk /inheritance:r /grant:r 'SYSTEM:(OI)(CI)F' 'BUILTIN\Administrators:(OI)(CI)F' | Out-Null +"box $env:COMPUTERNAME backup dir $bk" +foreach ($h in 'uswap-vdc-nam1.nam.zimmer.com','uswap-vdc-nam2.nam.zimmer.com') { foreach ($p in 636,389) { + $c = New-Object Net.Sockets.TcpClient; $ok = $c.ConnectAsync($h,$p).Wait(4000); $c.Dispose(); "reach $h`:$p $ok" } } +$root = @(Get-ChildItem Cert:\LocalMachine\Root | Where-Object { $_.Subject -like '*zimmer-rootca*' }) +"zimmer-rootca in LocalMachine\Root: $($root.Count) $(($root | % { $_.Thumbprint + ' exp ' + $_.NotAfter.ToString('yyyy-MM-dd') }) -join '; ')" +foreach ($h in 'uswap-vdc-nam1.nam.zimmer.com','uswap-vdc-nam2.nam.zimmer.com') { + try { $t = New-Object Net.Sockets.TcpClient($h,636); $script:err = $null + $s = New-Object Net.Security.SslStream($t.GetStream(), $false, { param($a,$b,$c,$e) $script:err = $e; $true }) + $s.AuthenticateAsClient($h); "tls $h policyErrors=$script:err proto=$($s.SslProtocol)"; $s.Dispose(); $t.Dispose() } catch { "tls $h FAILED $($_.Exception.Message)" } } +$svcs = Get-CimInstance Win32_Service | Where-Object { $_.PathName -match 'ApiInstall\\(HistorianGateway|MxAccessGateway|MxGateway|OtOpcUa|ScadaBridge)' -or $_.Name -match 'HistorianGateway|MxGateway|MxAccess|OtOpcUa|ScadaBridge' } +$show = '(?i)ENVIRONMENT$|DisableLogin|AllowOutside|Cookie|__Ldap__(Enabled|DevStub|Server|Port|Transport|AllowInsecure|SearchBase|ServiceAccountDn|UserNameAttribute|DisplayNameAttribute|GroupAttribute|BindMode|FallbackServers|UseTls|RequiredGroup)|GroupToRole|Secrets__.*(SqlitePath|Source|FilePath|EnvVarName|Provider|Enabled|Schema|Table|Mode)$|RequiredGroup|Thumbprint|Redundancy|PeerApplicationUris|ApplicationUri|^ScadaBridge__Node|Cluster__Roles|Runner|ASPNETCORE_URLS' +foreach ($s in $svcs) { + "== service $($s.Name) state=$($s.State) path=$($s.PathName)" + & reg.exe export "HKLM\SYSTEM\CurrentControlSet\Services\$($s.Name)" "$bk\reg-$($s.Name).reg" /y | Out-Null + $k = "HKLM:\SYSTEM\CurrentControlSet\Services\$($s.Name)" + $p = Get-ItemProperty "$k\Parameters" -ErrorAction SilentlyContinue + if ($p.AppDirectory) { " nssm AppDirectory=$($p.AppDirectory) Application=$($p.Application)" } + $lines = @((Get-ItemProperty $k -ErrorAction SilentlyContinue).Environment | % { "[svc] $_" }) + @($p.AppEnvironmentExtra | % { "[nssm] $_" }) | Where-Object { $_ } + foreach ($l in $lines) { $tag, $l = $l -split " ", 2; $i = $l.IndexOf('='); if ($i -lt 1) { continue }; $n = $l.Substring(0,$i) + if ($n -match $show -and $l -notmatch '(?i)password|key=|pepper|secret=') { " $tag env $l" } else { " $tag env $n=" } } +} +foreach ($p in 'HistorianGateway\app\appsettings*.json','MxAccessGateway\app\appsettings*.json','OtOpcUa\app\appsettings*.json','ScadaBridge\app\appsettings*.json') { + foreach ($f in Get-ChildItem "E:\ApiInstall\$p" -ErrorAction SilentlyContinue) { $d = "$bk\" + ($f.FullName.Substring(15) -replace '\\','_'); Copy-Item $f.FullName $d; "copied $($f.FullName) sha256 $((Get-FileHash $d).Hash.Substring(0,16))" } } +foreach ($d in 'HistorianGateway','MxAccessGateway','OtOpcUa','ScadaBridge') { foreach ($dll in Get-ChildItem "E:\ApiInstall\$d\app" -Filter '*.dll' -ErrorAction SilentlyContinue | Where-Object { $_.Name -match '^(ZB\.MOM\.WW\.(HistorianGateway\.Server|OtOpcUa\.Host|ScadaBridge\.Host)|MxGateway\.Server)\.dll$' }) { "ver $d $($dll.Name) $($dll.VersionInfo.ProductVersion)" } } +Get-ChildItem 'E:\ApiInstall' -Directory | % { "dir $($_.Name)" } +Get-ChildItem 'E:\ApiInstall\_tools' -Recurse -Filter '*.exe' -ErrorAction SilentlyContinue | % { "tool $($_.FullName) $($_.VersionInfo.ProductVersion)" } +foreach ($u in 'http://127.0.0.1:9000/health/active','http://127.0.0.1:8085/health/active','http://127.0.0.1:8084/health/active') { try { $r = Invoke-WebRequest -UseBasicParsing $u -TimeoutSec 5; "$u $($r.StatusCode) $($r.Content)" } catch { "$u $($_.Exception.Response.StatusCode.value__)" } } +"PREFLIGHT DONE $bk" diff --git a/docs/deploy/2026-11-19-qa-nam-ldap/sb-qa.tmpl.ps1 b/docs/deploy/2026-11-19-qa-nam-ldap/sb-qa.tmpl.ps1 new file mode 100644 index 0000000..29b8e83 --- /dev/null +++ b/docs/deploy/2026-11-19-qa-nam-ldap/sb-qa.tmpl.ps1 @@ -0,0 +1,131 @@ +# ScadaBridge QA (vq03/vq04) 061432c8 -> 2e0ce81e (#144/#145, no migration) + NAM AD LDAP, one window. Rendered per call: ACTION / TS. +# prep | backup (vq03) | stop-site | stop-central | swap | start-central | start-site | verify | rollback-swap +# Derived from ScadaBridge deploy-prep/2026-11-15-143/qa.tmpl.ps1 (migrate dropped; the swap also installs the per-node NAM +# appsettings.Central.json and removes the GLAuth password env entry from the Central service). +$ErrorActionPreference = 'Stop' +$ProgressPreference = 'SilentlyContinue' +$Action = '__ACTION__' +$ts = '__TS__' +$sha = '2e0ce81e' +$fromSha = '061432c8' +$zipSha256 = '6C706B0A76B7E9EC7CA00E12CEA19C5D86814BEAF66B541F527CBFC3D6572E23' +$stageDir = "E:\psrem-stage\sb-$sha" +$stage = "$stageDir\unpacked" +$root = 'E:\ApiInstall\ScadaBridge' +$log = "$root\_DEPLOY-NOTES-$sha-$ts.txt" +$settings = 'appsettings.json','appsettings.Central.json','appsettings.Site.json' +$centralKey = 'HKLM:\SYSTEM\CurrentControlSet\Services\ScadaBridge-Central' +$namCentral = "$stageDir\appsettings.Central.json.nam-" + $env:COMPUTERNAME.Substring($env:COMPUTERNAME.Length-4).ToLower() +$bak = "F:\MSSQL\BACKUP\ScadaBridge_pre-$($sha)_$ts.bak" +function Say($m) { $line = "[{0:HH:mm:ss.fff}] {1} {2}" -f (Get-Date), $env:COMPUTERNAME, $m; $line; Add-Content -Path $log -Value $line } +function Is-Dropped($e) { $e -like 'ScadaBridge__Security__Ldap__ServiceAccountPassword=*' } +function Digest($entries) { + $h = [System.Security.Cryptography.SHA256]::Create(); $o = @() + foreach ($e in $entries) { $i = $e.IndexOf('='); if ($i -lt 1) { $o += ''; continue } + $v = $e.Substring($i+1); $x = -join ($h.ComputeHash([Text.Encoding]::UTF8.GetBytes($v)) | % { $_.ToString('x2') }) + $o += ("{0} sha256:{1} len={2}" -f $e.Substring(0,$i), $x.Substring(0,12), $v.Length) } + $h.Dispose(); $o | Sort-Object +} +function Wait-Ready($url, $seconds) { + $deadline = (Get-Date).AddSeconds($seconds) + while ((Get-Date) -lt $deadline) { try { if ((Invoke-WebRequest -UseBasicParsing $url -TimeoutSec 5).StatusCode -eq 200) { return $true } } catch {}; Start-Sleep 2 } + return $false +} +function Stop-Svc($s) { + if ((Get-Service $s).Status -ne 'Stopped') { Say "stopping $s"; Stop-Service $s -Force; (Get-Service $s).WaitForStatus('Stopped','00:02:00') } + Say "$s stopped" +} +function Sql-Init { + $envC = @((Get-ItemProperty $centralKey).Environment) + $cs = ($envC | Where-Object { $_ -like 'ScadaBridge__Database__ConfigurationDb=*' }) -replace '^[^=]*=','' + $script:sb = New-Object System.Data.SqlClient.SqlConnectionStringBuilder $cs + $script:db = $sb['Database']; if (-not $db) { $script:db = $sb.InitialCatalog } + if ($db -ne 'ScadaBridge') { throw "unexpected catalog" } + $env:SQLCMDPASSWORD = $sb.Password +} +function Sql-Scalar($q) { $r = & sqlcmd -S $sb.DataSource -d $db -U $sb.UserID -C -h -1 -W -b -Q "SET NOCOUNT ON; $q"; if ($LASTEXITCODE -ne 0) { throw "sql failed" }; ($r | Where-Object { $_ -and $_.Trim() } | Select-Object -First 1).Trim() } +$schemaQ = "SELECT CONCAT((SELECT COUNT(*) FROM __EFMigrationsHistory),'|',(SELECT MAX(MigrationId) FROM __EFMigrationsHistory),'|',(SELECT COUNT_BIG(*) FROM dbo.AuditLog))" + +switch ($Action) { +'prep' { + Say "PREP $fromSha -> $sha" + if ((Get-FileHash "$stageDir\scadabridge-$sha.zip" -Algorithm SHA256).Hash -ne $zipSha256) { throw "zip hash mismatch" } + if (-not (Test-Path $namCentral)) { throw "missing $namCentral" } + Get-Content $namCentral -Raw | ConvertFrom-Json | Out-Null + if (Test-Path $stage) { Remove-Item $stage -Recurse -Force } + Expand-Archive "$stageDir\scadabridge-$sha.zip" $stage + $sv = (Get-Item "$stage\app\ZB.MOM.WW.ScadaBridge.Host.exe").VersionInfo.ProductVersion + $sc = (Get-Item "$stage\cli\scadabridge.exe").VersionInfo.ProductVersion + if ($sv -notlike "*+$sha*" -or $sc -notlike "*+$sha*") { throw "staged exe $sv / $sc" } + $lv = (Get-Item "$root\app\ZB.MOM.WW.ScadaBridge.Host.exe").VersionInfo.ProductVersion + if ($lv -notlike "*+$fromSha*") { throw "live exe $lv" } + Say "staged $sv; live $lv; app files $((Get-ChildItem "$stage\app" -Recurse -File).Count), cli $((Get-ChildItem "$stage\cli" -Recurse -File).Count)" + foreach ($s in 'ScadaBridge-Central','ScadaBridge-Site') { + & reg.exe export "HKLM\SYSTEM\CurrentControlSet\Services\$s" "$stageDir\_envbak-$s-$ts.reg" /y | Out-Null + if ($LASTEXITCODE -ne 0) { throw "reg export $s failed" } + $e = @((Get-ItemProperty "HKLM:\SYSTEM\CurrentControlSet\Services\$s").Environment) + Say ("$s env entries {0}" -f $e.Count); Digest $e | % { Say " env $s $_" } + } + $keep = New-Item -ItemType Directory -Force -Path "$root\_appsettings-keep-$ts" + foreach ($f in $settings) { Copy-Item "$root\app\$f" "$keep\$f"; Say ("keep {0} sha256 {1}" -f $f, (Get-FileHash "$keep\$f" -Algorithm SHA256).Hash) } +} +'backup' { + Sql-Init + $pre = Sql-Scalar $schemaQ; Say "schema before: $pre" + if (-not $pre.StartsWith('65|')) { throw "expected 65 migrations" } + & sqlcmd -S $sb.DataSource -d master -U $sb.UserID -C -b -Q "BACKUP DATABASE [$db] TO DISK = N'$bak' WITH COPY_ONLY, CHECKSUM, COMPRESSION, INIT" | Out-Null + if ($LASTEXITCODE -ne 0) { throw "backup failed" } + & sqlcmd -S $sb.DataSource -d master -U $sb.UserID -C -b -Q "RESTORE VERIFYONLY FROM DISK = N'$bak' WITH CHECKSUM" | Out-Null + if ($LASTEXITCODE -ne 0) { throw "verifyonly failed" } + Say "backup $bak VERIFYONLY ok" +} +'stop-site' { Stop-Svc 'ScadaBridge-Site' } +'stop-central' { + Stop-Svc 'ScadaBridge-Central' + $deadline = (Get-Date).AddSeconds(30) + while ((Get-Process -Name 'ZB.MOM.WW.ScadaBridge.Host' -ErrorAction SilentlyContinue) -and (Get-Date) -lt $deadline) { Start-Sleep 1 } + if (Get-Process -Name 'ZB.MOM.WW.ScadaBridge.Host' -ErrorAction SilentlyContinue) { throw "Host.exe still running" } + Say "no Host.exe remains" +} +'swap' { + if (Get-Process -Name 'ZB.MOM.WW.ScadaBridge.Host' -ErrorAction SilentlyContinue) { throw "Host.exe running" } + Rename-Item "$root\app" "app.bak.$ts"; Rename-Item "$root\cli" "cli.bak.$ts" + Say "renamed app/cli -> *.bak.$ts" + Copy-Item "$stage\app" "$root\app" -Recurse + Copy-Item "$stage\cli" "$root\cli" -Recurse + $keep = "$root\_appsettings-keep-$ts" + foreach ($f in $settings) { + Copy-Item "$keep\$f" "$root\app\$f" -Force + if ((Get-FileHash "$keep\$f" -Algorithm SHA256).Hash -ne (Get-FileHash "$root\app\$f" -Algorithm SHA256).Hash) { throw "appsettings mismatch $f" } + Say "restored $f (sha256 match)" + } + Copy-Item $namCentral "$root\app\appsettings.Central.json" -Force + Say ("installed NAM appsettings.Central.json sha256 {0}" -f (Get-FileHash "$root\app\appsettings.Central.json" -Algorithm SHA256).Hash) + Say ("live Host " + (Get-Item "$root\app\ZB.MOM.WW.ScadaBridge.Host.exe").VersionInfo.ProductVersion + " / CLI " + (Get-Item "$root\cli\scadabridge.exe").VersionInfo.ProductVersion) + $before = @((Get-ItemProperty $centralKey).Environment) + $keepEnv = @($before | Where-Object { -not (Is-Dropped $_) }) + Set-ItemProperty -Path $centralKey -Name Environment -Value ([string[]]$keepEnv) -Type MultiString + $after = @((Get-ItemProperty $centralKey).Environment) + foreach ($e in $keepEnv) { if ($after -notcontains $e) { throw "env entry changed" } } + if (@($after | ? { Is-Dropped $_ }).Count -ne 0) { throw "GLAuth password entry still present" } + Say ("central env entries {0} -> {1}" -f $before.Count, $after.Count) +} +'start-central' { Say "starting ScadaBridge-Central"; Start-Service 'ScadaBridge-Central'; $ok = Wait-Ready 'http://localhost:8085/health/ready' 180; Say "central ready=$ok"; if (-not $ok) { throw "central not ready" } } +'start-site' { Say "starting ScadaBridge-Site"; Start-Service 'ScadaBridge-Site'; $ok = Wait-Ready 'http://localhost:8084/health/ready' 180; Say "site ready=$ok"; if (-not $ok) { throw "site not ready" } } +'verify' { + foreach ($s in 'ScadaBridge-Central','ScadaBridge-Site') { Say ("$s " + (Get-Service $s).Status) } + if ($env:COMPUTERNAME -eq 'WONDER-APP-VQ03') { Sql-Init; Say ("schema now: " + (Sql-Scalar $schemaQ)) } +} +'rollback-swap' { + foreach ($s in 'ScadaBridge-Site','ScadaBridge-Central') { Stop-Svc $s } + if (Test-Path "$root\app") { Rename-Item "$root\app" "app.failed.$ts" } + if (Test-Path "$root\cli") { Rename-Item "$root\cli" "cli.failed.$ts" } + Rename-Item "$root\app.bak.$ts" 'app'; Rename-Item "$root\cli.bak.$ts" 'cli' + & reg.exe import "$stageDir\_envbak-ScadaBridge-Central-$ts.reg" | Out-Null + Say "rolled back binaries + Central env (services left STOPPED; start Central vq03, vq04, then Sites)" +} +default { throw "bad action $Action" } +} +Remove-Item Env:SQLCMDPASSWORD -ErrorAction SilentlyContinue +Say "$Action OK" +exit 0 diff --git a/docs/deploy/2026-11-19-qa-nam-ldap/seed-secret.ps1 b/docs/deploy/2026-11-19-qa-nam-ldap/seed-secret.ps1 new file mode 100644 index 0000000..b58b23b --- /dev/null +++ b/docs/deploy/2026-11-19-qa-nam-ldap/seed-secret.ps1 @@ -0,0 +1,46 @@ +# Seed nam-ldap-svc-password into every QA app store on THIS box, reading each store's location and KEK from the +# service's own env (never assumed). ScadaBridge Central's store is the shared SQL Server store: seeded once, on vq03, +# with a CLI variant that honours Secrets:Store:Provider=SqlServer. The value arrives in $vf over SMB and is deleted +# at the end; it is never printed. Usage: rendered with __BK__ (the box's protected _ldapcut dir). +$ErrorActionPreference = 'Stop' +$bk = '__BK__' +$vf = "$bk\_v.tmp" +$cli = 'E:\ApiInstall\_tools\secret-0.6.6\ZB.MOM.WW.Secrets.Cli.exe' +$cliSql = 'E:\ApiInstall\_tools\secret-0.6.6-sql\ZB.MOM.WW.Secrets.Cli.exe' +$name = 'nam-ldap-svc-password' +$desc = 'NAM LDAP lookup account (svc_igntadmin, QA interim 2026-11-19)' +& icacls.exe $vf /inheritance:r /grant:r 'BUILTIN\Administrators:F' 'NT AUTHORITY\SYSTEM:F' | Out-Null +function SvcEnv($svc) { + $k = "HKLM:\SYSTEM\CurrentControlSet\Services\$svc" + $l = @((Get-ItemProperty $k -ErrorAction SilentlyContinue).Environment) + @((Get-ItemProperty "$k\Parameters" -ErrorAction SilentlyContinue).AppEnvironmentExtra) | Where-Object { $_ } + $h = @{}; foreach ($e in $l) { $i = $e.IndexOf('='); if ($i -gt 0) { $h[$e.Substring(0,$i)] = $e.Substring($i+1) } }; $h +} +$targets = @('HistorianGateway','MxAccessGw','OtOpcUaHost') +if ($env:COMPUTERNAME -eq 'WONDER-APP-VQ03') { $targets += 'ScadaBridge-Central' } +$names = 'Secrets__SqlitePath','Secrets__MasterKey__Source','Secrets__MasterKey__FilePath','Secrets__MasterKey__EnvVarName','Secrets__Store__Provider','Secrets__SqlServer__ConnectionString','ZB_SECRETS_MASTER_KEY' +$fail = 0 +try { + $want = [IO.File]::ReadAllText($vf) + "value file read, length $($want.Length)" + foreach ($svc in $targets) { + $e = SvcEnv $svc + foreach ($n in $names) { Remove-Item "Env:$n" -ErrorAction SilentlyContinue } + foreach ($n in $names) { if ($e.ContainsKey($n)) { Set-Item "Env:$n" $e[$n] } } + $exe = $cli + if ($svc -eq 'ScadaBridge-Central') { $env:Secrets__Store__Provider = 'SqlServer'; $exe = $cliSql + if (-not $env:Secrets__SqlServer__ConnectionString) { throw 'Central env has no Secrets__SqlServer__ConnectionString' } } + if ($env:Secrets__MasterKey__Source -ne 'File') { "$svc master key source '$($env:Secrets__MasterKey__Source)' - not File, skipping"; $fail++; continue } + $where = if ($svc -eq 'ScadaBridge-Central') { 'SqlServer shared store' } else { $env:Secrets__SqlitePath } + $out = & cmd.exe /c "`"$exe`" set $name - --description `"$desc`" < `"$vf`"" 2>&1 + $rc = $LASTEXITCODE + $got = (& $exe get $name 2>$null | Out-String).TrimEnd("`r","`n") + $match = ($got -ceq $want) + if (-not $match) { try { $j = $got | ConvertFrom-Json; $match = ($j.value -ceq $want) } catch {} } + "$svc -> $where kek=$($env:Secrets__MasterKey__FilePath) set rc=$rc out=$(($out | Out-String).Trim()) readback-match=$match" + if ($rc -ne 0 -or -not $match) { $fail++ } + } +} finally { + Remove-Item $vf -Force -ErrorAction SilentlyContinue; "value file removed: $(-not (Test-Path $vf))" + foreach ($n in $names) { Remove-Item "Env:$n" -ErrorAction SilentlyContinue } +} +"SEED DONE failures=$fail"; exit $fail diff --git a/docs/plans/2026-11-15-nam-ldap-cutover.md b/docs/plans/2026-11-15-nam-ldap-cutover.md index 2d16c35..eee0542 100644 --- a/docs/plans/2026-11-15-nam-ldap-cutover.md +++ b/docs/plans/2026-11-15-nam-ldap-cutover.md @@ -1,4 +1,4 @@ -# NAM AD LDAP cutover — DEV vd03, then QA vq03/vq04 (plan, NOT executed) +# NAM AD LDAP cutover — DEV vd03, then QA vq03/vq04 (DEV + QA EXECUTED 2026-11-19 on the interim account) > **Status lives in Gitea.** Execution is tracked by > [scadaproj#6](https://gitea.dohertylan.com/dohertj2/scadaproj/issues/6). The dedicated, shared LDAP-lookup service @@ -119,7 +119,10 @@ warning, not the second. Ask IT for the account's lockout state before any retry --- -## 0Q. QA vq03/vq04, with today's groups and accounts (plan 2026-11-19, NOT executed) +## 0Q. QA vq03/vq04, with today's groups and accounts (plan 2026-11-19; EXECUTED 2026-11-19) + +**Executed 2026-11-19 15:29–16:10 EST:** all four apps on both boxes on the DEV builds and NAM AD; record +[`docs/deploy/2026-11-19-qa-nam-ldap/`](../deploy/2026-11-19-qa-nam-ldap/README.md). Optional step 8 (oswaltk) not run. **Owner decisions (2026-11-19):** 1. QA binds as **`svc_igntadmin`** as well (interim). This supersedes A6 and the §7 note "QA never runs on