Files
mxaccessgw/docs/deploy/2026-11-19-vd03-qa-e74918cc-nam-ldap.md
T
Joseph Doherty 57d5b7c364
ci / java (push) Failing after 21m8s
ci / windows-x86 (push) Failing after 21m18s
ci / portable (push) Failing after 25m33s
docs(deploy): DEV vd03 + QA vq03/vq04 on e74918cc, dashboard login on NAM AD
Claude-Session: https://claude.ai/code/session_01GT67AD5mc2BnaaAfY3hqqQ
2026-11-19 16:21:31 -05:00

5.2 KiB

2026-11-19: DEV vd03 + QA vq03/vq04 on e74918cc, dashboard login on NAM AD

Server and x86 Worker moved from 76a71bc8 to main e74918cc (dashboard styling, accessible Galaxy browse/watch, overview fault context, Telemetry 0.3.1; no proto change), then dashboard login moved to NAM AD as part of the family cutover (scadaproj plan docs/plans/2026-11-15-nam-ldap-cutover.md §0 DEV, §0Q QA). Owner approvals 2026-11-19: "ok, do it. winrm usage approved (for the vd03 deploy)" and "do the qa ldap plan, winrm usage approved". Family records: scadaproj docs/deploy/2026-11-19-vd03-nam-ldap/ and docs/deploy/2026-11-19-qa-nam-ldap/. No rollback.

Package mxgw-e74918cc.zip sha256 f0999b75bee196375aeb8e3e277a0e948c54a747fd5251d336461c9665d847f2 (Server publish + Worker rebuilt on windev at e74918cc, -p:Platform=x86, apphost renamed MxGateway.Worker.exe for the wonder naming). ProductVersion 0.1.2+e74918cc on both. Bundled runtime 10.0.14 = live
Staged vd03 E:\psrem-stage\mxgw-ui-e74918cc\; QA E:\psrem-stage\mxgw-e74918cc\ (the same zip pulled from vd03 over SMB, re-hashed on each box)
Script scripts/deploy/Upgrade-MxGatewayService.ps1 (4890a3d1) with one local patch: nssm start wrapped in try/catch, so a benign SERVICE_START_PENDING reply no longer aborts and auto-rolls back; the script's own 60 s RUNNING wait decides. The repo script is not fixed yet

DEV wonder-app-vd03

  • 13:43 and 13:46: two failed attempts. Both aborted at nssm start on Unexpected status SERVICE_START_PENDING and rolled back automatically; the old build came back healthy each time (the second printed a misleading "AUTOMATIC ROLLBACK FAILED"). Leftovers E:\ApiInstall\MxGateway\{Server,Worker}.failed-20261119-1343*/1346* can be removed.
  • 13:47:38 patched run: carried appsettings.json, appsettings.Development.json, dashboard.pfx; env block (9 entries, b697148e6cbc) unchanged at every step; /health/ready and /healthz 200; deployed SHA e74918cc; 60 s stability window clean; 2 of 2 MXAccess sessions ready, 0 faulted (existing consumer keys work through the new Worker). Downtime ~10 s on the final run. Backups Server.bak-/Worker.bak-20261119-134738-ui-e74918cc, _deploy-backup-20261119-134738-ui-e74918cc\.
  • 15:10 NAM login. The service runs ASPNETCORE_ENVIRONMENT=Staging with no overlay, so Server\appsettings.json is the file read (the root MxGateway\appsettings.json is a stale unused copy). Edit: NAM MxGateway:Ldap block (LDAPS uswap-vdc-nam1…:636, fallback nam2:636, svc_igntadmin@nam.zimmer.com, ${secret:nam-ldap-svc-password}, sAMAccountName/name/memberOf, BindMode=ServiceAccount; stale RequiredGroup dropped), DisableLogin=false, DEV GroupToRole (UG_WW_APP_ADMIN_DEV→Administrator, UG_WW_APP_IDE_DEV→Viewer, UG_WW_APP_IDE_RO_DEV→Viewer) replacing the SCADA-* rows; restart. Gates: anonymous → /login; bogus refused; dohertj2 signs in as Administrator.

QA wonder-app-vq03 + vq04 (one window: the upgrade and the LDAP change share one restart)

On QA the LDAP settings live in the NSSM env (MxGateway__Ldap__*, GLAuth 127.0.0.1:3893), not in the file.

  1. Secret nam-ldap-svc-password seeded into data\secrets.db (KEK C:\ProgramData\ZB\kek\mxgateway.kek), read back.
  2. env-mxgw.ps1 (33 → 38 entries): the eight GLAuth MxGateway__Ldap__* entries and three …GroupToRole__SCADA-* entries replaced by the NAM block as MxGateway__Ldap__* plus MxGateway__Dashboard__GroupToRole__UG_WW_APP_ADMIN_QA=Administrator, …UG_WW_APP_IDE_QA=Viewer, …UG_WW_APP_IDE_RO_QA=Viewer. MxGateway__Dashboard__RequireHttpsCookie=false kept.
  3. Base app\appsettings.json: the Dashboard:GroupToRole GwAdmin/GwReader pair removed (it would merge with the env map). New sha 778534b9e771…; the upgrade script carried it into the new app\, so future upgrades keep it.
  4. Patched upgrade script (dry run first on vq03): vq03 stamp 20261119-155641, vq04 20261119-155819. Env fingerprint unchanged after stop, swap, start and the 60 s window on both (vq03 239b02d0dcd8, vq04 48aae910c944); /health/ready 200; deployed SHA e74918cc.

Gates (both boxes): anonymous → 302 /login; zz_bogus_user_ldapcut refused; dohertj2 → 302 / (Administrator). Worker path with pre-existing keys: ScadaBridge's MxGateway connection reconnected after its own restart and reads 2196/2196 Good. No LDAP or service-bind warning in the OTel log export.

Rollback

  • Binaries: Upgrade-MxGatewayService.ps1 -Rollback -BackupStamp <stamp> (stamps above).
  • LDAP: vd03 — restore E:\ApiInstall\_ldapcut-20261119T145640\MxGateway_Server_appsettings.json; QA — reg import E:\ApiInstall\_ldapcut-20261119T152947\reg-MxAccessGw.reg (vq03) / …T152958\… (vq04) and restore MxAccessGateway_app_appsettings.json from the same folder. Restart.

Follow-ups

  • Fix Start-Gateway in Upgrade-MxGatewayService.ps1 (the START_PENDING abort) in the repo.
  • Every redeploy must keep the NAM settings: QA keeps them in env (untouched by the script); vd03 keeps them in Server\appsettings.json, which the script carries forward.