Files
scadaproj/docs/deploy/2026-11-19-prod-stage

2026-11-19: PROD install pack staged on wonder-app-vp03/vp04 + wonder-par-vp03/vp04 (SMB only)

Owner request (2026-11-19): "prep the application files, scripts, configs and copy them to the 4 servers for prod". Plan: docs/plans/2026-11-19-prod-scadabridge-mxgw-install.md.

What was done: files copied over SMB (445) as NAM\adm_dohertj2. Nothing ran on any PROD box: no WinRM, no service, no install, no KEK, no secret. The PROD deploy hold and every phase gate in the plan still apply. Each script below is run on the box only after the owner's go for its phase.

Artifacts (the QA-soaked builds, byte-identical; plan D-P2)

File sha256 Source
scadabridge-2e0ce81e.zip (app + cli, self-contained win-x64) 6c706b0a…6572e23 the zip QA vq03/vq04 run since 2026-11-19
mxgw-e74918cc.zip (Server + x86 Worker MxGateway.Worker.exe, 0.1.2+e74918cc) f0999b75…5d847f2 the zip DEV vd03 + QA run
otelcol-contrib_0.162.0_windows_amd64.zip 97a675ee…e496a167 QA vq03 E:\psrem-stage\otel-54 (scadaproj#54)
Secrets CLI 0.6.6 ZB.MOM.WW.Secrets.Cli.exe 36dbd512… = QA E:\ApiInstall\_tools\secret-0.6.6
Secrets CLI 0.6.6 + SqlServer provider (centrals only) 421f95e9… = QA …\secret-0.6.6-sql (QA finding 2)
nssm.exe eee9c44c… = QA C:\ProgramData\ZB\qa-build\tools\nssm.exe

Every staged folder carries a SHA256SUMS.txt. Remote sizes were checked file by file over SMB (150/150), and the two app zips plus the CLI were pulled back from vp04 and par-vp04 and re-hashed.

Layout on each box

Path vp03 / vp04 par-vp03 / par-vp04
E:\psrem-stage\prod-common\ nssm.exe, preflight-prod.ps1, seed-secret-prod.ps1 same
E:\psrem-stage\mxgw-e74918cc\ zip, appsettings.json.prod, mxgw.env, install-mxgw.ps1, Install-/Upgrade-MxGatewayService.ps1 same (install refuses until MXAccess exists, Q3)
E:\psrem-stage\sb-2e0ce81e\ zip, appsettings.Central.json.prod-<box>, sb-central.env, install-sb.ps1, sql\ zip, appsettings.Site.json.prod-<box>, sb-site.env, install-sb.ps1
E:\psrem-stage\otel-54\ collector zip, config.yaml, Install-OtelCollector.ps1 (PROD tier added), Test-OtelCollector.ps1 same
E:\ApiInstall\_tools\ secret-0.6.6\, secret-0.6.6-sql\ secret-0.6.6\

E:\ApiInstall now exists only for _tools; there is still no app folder and no service, so both install scripts see a greenfield box.

Configs (sources in this folder)

  • mxgw mxgw.env: QA vq03's key set plus the §3A.3 LDAP block. Differences: ASPNETCORE_ENVIRONMENT=Production (login hard-stops on), _PROD GroupToRole, and the file KEK C:\ProgramData\ZB\kek\mxgateway.kek. appsettings.json.prod = the zip's Server\appsettings.json minus Dashboard:GroupToRole (GwAdmin/GwReader).
  • ScadaBridge centrals: per node appsettings.Central.json.prod-vp03|vp04 (QA shape, IPv4 literals, self-first seeds, leader-auto-down, BootstrapGuard on, NAM LDAPS block, RequireHttpsCookie=false, shared SQL secret + API-key stores, CentralGrpcPort left at the default 8083: QA's 8086 only existed because of co-location). sb-central.env is QA's key set minus MachineDataDb (gone since 3808b49f), minus the LDAP password env (§3A.3), and minus SitePsks (central reads SB-GRPC-PSK-<siteId> from its store, Phase 6).
  • ScadaBridge sites: appsettings.Site.json.prod-parvp03|parvp04 (node-a/node-b, PeerAddress on node-a only, both centrals at :8083, absolute data paths). sb-site.env = QA's site key set.
  • SQL (centrals, sql\): scadabridge-schema-2e0ce81e.sql (full, non-idempotent, 65 migrations, from dotnet ef migrations script 0 20261115221558_RemoveAuditLogHashChain at 2e0ce81e; run with sqlcmd -I -b), zbsecrets-schema-v1.sql, sb-role-mappings-prod.sql (the five _PROD rows, D-P6).

Placeholders still open (the install scripts refuse while any remains)

Placeholder / value Where Question
__Q2_ALARM_EXPRESSION__ mxgw.env Q2 alarm area (\\<GR node>\Galaxy!<area>)
MxGateway__Galaxy__ConnectionString (value file) install time Q2 prod GR SQL
__Q8_SITE_ID__ both appsettings.Site.json.prod-* Q8 SiteId
ScadaBridge__Database__ConfigurationDb, Secrets__SqlServer__ConnectionString (value files) install time Q1 prod SQL + scadabridge_svc
JWT key, inbound pepper (same on both centrals), mxgw pepper (shared by the PAR pair, pattern a), site PSK + LocalDb key (same on both sites) install time, value files generated then, stored in the Keychain

When a placeholder is resolved, edit the file here, commit, and re-push that one file (its SHA256SUMS.txt line too).

Scripts (dry first, always)

Script Does Never does
prod-common\preflight-prod.ps1 [-SqlHost] [-GrHost] read-only Phase 2 gates: disk, LmxProxy.dll + COM registration, NAM 636/389 + rootca + TLS, greenfield paths and ports, DNS, TCP path probes per §4 (open / refused / timeout), staged hashes write anything
mxgw-e74918cc\install-mxgw.ps1 -Mode dry|real [-ValuesDir] [-KekFrom] MXAccess gate, zip hash, placeholder + _DEV/_QA refusal, expand to app/worker, NAM base config, 32-byte KEK, NSSM service via the patched Install-MxGatewayService.ps1 start the service
sb-2e0ce81e\install-sb.ps1 -Role Central|Site -Mode dry|real [-ValuesDir] [-KekFrom] box/role check, zip hash, this node's config, env from value files, expand app/cli, <role>\{data,logs}, KEK (generated on vp03 only; every other node takes the same file), sc.exe service with recovery 5/5/10 s + failureflag start the service
prod-common\seed-secret-prod.ps1 -Target MxAccessGw|ScadaBridge-Central -ValueFile seeds nam-ldap-svc-password into a stopped service's store (central: once, vp03, shared SQL store), reads back, deletes the value file print the value

Install-MxGatewayService.ps1 here is mxgw e74918cc plus three marked patches: -WorkerExeName (the wonder Worker is MxGateway.Worker.exe), File-KEK acceptance, and masking of connection strings in its env printout (the stock script prints MxGateway__Galaxy__ConnectionString in clear). Upgrade-MxGatewayService.ps1 is the START_PENDING-patched copy QA used. Both repo fixes are still owed (mxaccessgw).

Checked on the Mac (pwsh 7, stubs for nssm/icacls): all eight scripts parse; install-sb.ps1 dry runs pass for a central and a site and refuse a wrong role and an open __Q8_SITE_ID__; install-mxgw.ps1 dry run refuses an open __Q2__ and then runs Install-MxGatewayService -DryRun (39 env entries). That is not a Windows run; the on-box dry run is the gate.

Order once the owner says go (plan phases)

  1. Phase 2, each box: preflight-prod.ps1 (needs the Q6 go for remote exec). Optional: otel-54\Install-OtelCollector.ps1 -StageDir … -Tier PROD -NssmPath E:\psrem-stage\prod-common\nssm.exe.
  2. Phase 3 (Q1 answered): sql\scadabridge-schema-2e0ce81e.sql, sql\zbsecrets-schema-v1.sql, sql\sb-role-mappings-prod.sql; backup.
  3. Phase 4, vp03/vp04 (par later, Q3): install-mxgw.ps1 -Mode dry, then real; seed-secret-prod.ps1 -Target MxAccessGw; start; §3A.5.
  4. Phase 5: install-sb.ps1 -Role Central on vp03 (KEK generated), vp04 (-KekFrom the vp03 file); seed-secret-prod.ps1 -Target ScadaBridge-Central on vp03; start vp03, then vp04; §3A.5.
  5. Phase 6: install-sb.ps1 -Role Site on both PAR boxes (-KekFrom, same PSK and LocalDb key); SB-GRPC-PSK-<siteId> into central; start both together.

Cleanup (if PROD is abandoned)

winrm smb rm -r <box> 'E$\psrem-stage\prod-common' 'E$\psrem-stage\mxgw-e74918cc' 'E$\psrem-stage\sb-2e0ce81e' 'E$\psrem-stage\otel-54' 'E$\ApiInstall\_tools' on each box. The push also created the parent folders E:\psrem-stage and E:\ApiInstall; the earlier 2026-11-19 SMB survey listed neither. Nothing else was created.