84 lines
6.9 KiB
Markdown
84 lines
6.9 KiB
Markdown
# 2026-11-19: DEV vd03 NAM AD LDAP cutover (HistorianGateway, MxAccessGateway, ScadaBridge, OtOpcUa)
|
||
|
||
**Owner go (2026-11-19):** "ok, do it. winrm usage approved (for the vd03 deploy)". Plan:
|
||
[`docs/plans/2026-11-15-nam-ldap-cutover.md`](../../plans/2026-11-15-nam-ldap-cutover.md) §0 (today's groups and accounts).
|
||
Umbrella: scadaproj#6. **DEV only; QA untouched (hold stands).**
|
||
|
||
**Result: all four apps on vd03 now sign users in against NAM AD** over LDAPS (nam1, nam2 as fallback), bound as
|
||
`svc_igntadmin@nam.zimmer.com` with the password from `${secret:nam-ldap-svc-password}` in each app's own store.
|
||
Zero service-account bind failures during the whole run.
|
||
|
||
## Steps (EST)
|
||
|
||
| Time | Step |
|
||
|---|---|
|
||
| 14:52–14:53 | **ScadaBridge `061432c8` → `2e0ce81e`** (#144 + #145, binary-only, no migration). Record: ScadaBridge `docs/deployment/deployment-records/2026-11-19-wonder-app-vd03-2e0ce81e.md`. Site ~47 s, central API ~30 s down |
|
||
| 14:56 | Phase 0 (`preflight.ps1`): nam1/nam2 reachable on 636 + 389 from vd03; **`zimmer-rootca` already in `LocalMachine\Root`** (thumbprint `78D941D7…`, exp 2036), LDAPS chain validates with no policy errors → `AllowInsecure=false` everywhere (D4 settled). Backups of every service key and appsettings → `E:\ApiInstall\_ldapcut-20261119T145640\`. OtOpcUa `LdapGroupRoleMapping` overlay: 0 rows |
|
||
| 14:58 | Secret seeded in all four stores (`seed-secret.ps1`) with the Secrets CLI built from current source (0.6.6, stdin input; the box's 0.6.2 CLI has no stdin mode). No schema or envelope change since 0.6.2. Read-back matched in all four; the transient value file was ACL'd to Administrators/SYSTEM and deleted. CLI left at `E:\ApiInstall\_tools\secret-0.6.6\` |
|
||
| 15:01 | **HistorianGateway**: `Server\appsettings.Production.json` → NAM block + `Dashboard:DisableLogin=false` + DEV `GroupToRole`; restart |
|
||
| 15:10 | **MxAccessGateway**: `Server\appsettings.json` (the service runs `ASPNETCORE_ENVIRONMENT=Staging`, no overlay; the root `MxGateway\appsettings.json` is a stale copy, unused) → NAM block (stale `RequiredGroup` dropped) + `DisableLogin=false` + DEV `GroupToRole` (replaces the `SCADA-*` rows); restart |
|
||
| 15:11 | **ScadaBridge Central**: `appsettings.Central.json` → NAM block; `Auth` block (`DisableLogin`/`AllowOutsideDevelopment`) removed; env entry `ScadaBridge__Security__Ldap__ServiceAccountPassword` (GLAuth) removed so it can't override the secret (12 → 11 entries); Central restarted, Site untouched |
|
||
| 15:12 | ScadaBridge role rows by SQL on the box (`sb-role-mappings.ps1`; nobody held a role yet): ids 6–9 |
|
||
| 15:14 | **OtOpcUa**: 18 `Security__Ldap__*` env entries + `Security__Auth__DisableLogin` true→false; restart (~9 s) |
|
||
| 15:15 | OtOpcUa: `Security__Cookie__RequireHttpsCookie=false` added; restart (~9 s). See finding 1 |
|
||
|
||
## Mappings now on vd03
|
||
|
||
| Group | OtOpcUa | ScadaBridge | MxAccessGateway | HistorianGateway |
|
||
|---|---|---|---|---|
|
||
| `UG_WW_APP_ADMIN_DEV` | Administrator | Administrator | Administrator | Administrator |
|
||
| `UG_WW_APP_IDE_DEV` | Designer | Designer | Viewer | — |
|
||
| `UG_WW_APP_IDE_RO_DEV` | Viewer | Viewer | Viewer | Viewer |
|
||
| `UG_WW_HIST_ADMIN_DEV` | Viewer | **Deployer (DEV interim, plan §0.2)** | — | Administrator |
|
||
|
||
ScadaBridge still holds the five `SCADA-*` seed rows (ids 1–5); they name GLAuth groups and grant nothing in NAM.
|
||
|
||
## Verification
|
||
|
||
| Check | HG `:5220` | mxgw `:5130` | ScadaBridge `:8085` | OtOpcUa `:9000` |
|
||
|---|---|---|---|---|
|
||
| Anonymous `GET /` → login | 302 | 302 | 302 | 302 |
|
||
| Bogus user refused, no `system-side` warning | ✓ | ✓ | ✓ | ✓ |
|
||
| `dohertj2` signs in | ✓ | ✓ | ✓ cookie; `/auth/token` 200 roles Administrator+Designer+Deployer; CLI (HTTP Basic) `instance list` + admin `role-mapping list` | ✓ cookie, `GET /` 200; `/auth/token` 200 roles Viewer+Designer+Administrator |
|
||
| Zero-role user refused | ✓ (`adm_dohertj2`, "no GroupToRole mapping matched") | not run (already refused by code) | ✓ `dohertj2` **before** the rows existed: `/auth/login`, `/auth/token` and the CLI all refused, log "bind succeeded but none of the user's 176 group(s) maps to a ScadaBridge role" | not run (needs a second outage; covered by #913's tests) |
|
||
| Health after | ready 200 | ready 200 | central + site ready 200; MxGateway Connected, 810/810 resolved, good 399 / bad 411 (baseline 398/412) | ready 200 |
|
||
|
||
Overview `:5320` 200 throughout.
|
||
|
||
**Not done, by owner instruction:** the DC-failover checks (§7.6 step 5, acceptance A2). Before that instruction one HG
|
||
check had run: with the primary pointed at closed port nam1:637 the login went through nam2 and the library logged
|
||
`sticky preference moved to uswap-vdc-nam2…:636`. The config was then restored (byte-compared). A temporary Windows Firewall rule
|
||
was also tried and removed; it had no effect because the firewall is **off** on every profile on vd03.
|
||
|
||
**Viewer-only login was not tested live**: the only `APP_IDE_RO_DEV` members also hold `APP_ADMIN_DEV`.
|
||
|
||
## Findings
|
||
|
||
1. **OtOpcUa AdminUI login loops on plain HTTP.** `Security:Cookie:RequireHttpsCookie` defaults to `true`, so the
|
||
auth cookie is `Secure` and a browser never sends it back over `http://…:9000`. It never showed before because
|
||
login was off on vd03. Fixed for DEV with `Security__Cookie__RequireHttpsCookie=false`, as HG, mxgw and ScadaBridge
|
||
already run on vd03. QA needs the same decision (or HTTPS) before its OtOpcUa step.
|
||
2. **The box's Secrets CLI (0.6.2) can only take a value on argv.** Seeding needs the newer CLI (stdin), or the app's
|
||
`/admin/secrets` page. QA will need the same.
|
||
3. **mxgw `appsettings.json` exists twice** (`MxGateway\` and `MxGateway\Server\`); only `Server\` is read (NSSM
|
||
`AppDirectory`). The mxgw deploy notes must carry the `GroupToRole`/LDAP edit forward on every redeploy, since a
|
||
redeploy replaces `Server\appsettings.json`.
|
||
4. **HG's own log files are empty**; its logs go only to the box OTel collector (`E:\ApiInstall\OtelCollector\data\logs.jsonl`).
|
||
|
||
## Rollback (per app)
|
||
|
||
Everything is in `E:\ApiInstall\_ldapcut-20261119T145640\` (`reg-<service>.reg`, `<path>_appsettings*.json`):
|
||
- **HG / mxgw:** copy back the backed-up `Server\appsettings*.json`, restart the service.
|
||
- **ScadaBridge:** copy back `sb-appsettings.Central.json.pre-cutover` as `app\appsettings.Central.json`,
|
||
`reg import reg-ScadaBridge-Central.reg`, restart Central. The `UG_WW_*` rows can stay (they grant nothing with login off).
|
||
- **OtOpcUa:** `reg import reg-OtOpcUaHost.reg`, restart `OtOpcUaHost`.
|
||
- ScadaBridge binary rollback to `061432c8`: see the ScadaBridge record.
|
||
|
||
The seeded secrets are inert once nothing references them. The box-local GLAuth (`ScadaBridge-LDAP`) is still running.
|
||
|
||
## Files
|
||
|
||
`preflight.ps1`, `seed-secret.ps1`, `sb-central-cutover.ps1`, `sb-role-mappings.ps1`, `otopcua-cutover.ps1`,
|
||
`otopcua-cookie.ps1`, `ldap-block.json` (the block written into HG, mxgw and ScadaBridge), and `login-test.sh` (Mac-side
|
||
login probe; passwords come from the Keychain into a 0600 temp file, never argv).
|