mxaccessgw/docs/runbooks/SEC-36-ldap-credential-rotation.md

SEC-36 — LDAP Service-Account Credential Rotation (Operator Runbook)

Operator steps to rotate the shared GLAuth service-account password after the repo-side removal landed (SEC-36). The repo change (removal of the committed value, the two supported secret channels, and this runbook) is already merged; the live rotation below is the load-bearing half and is yours to execute.

Never put the old or new password in this repo, in a commit, in a chat, or in this file. The value lives only in the GLAuth source of truth and in each host's out-of-band channel.

Why

The dev GLAuth service-account password (cn=serviceaccount,dc=zb,dc=local) was historically committed to this repo. Removal alone is insufficient — the old value is permanently recoverable from git history — so rotation is required. Until the shared GLAuth on 10.100.0.35:3893 stops honoring the old value, the repo history discloses a live directory account with LDAP search capability over dc=zb,dc=local.

Where the credential lives now (three channels, all bind MxGateway:Ldap:ServiceAccountPassword)

• Source of truth: scadaproj/infra/glauth/config.toml on host 10.100.0.35 (the serviceaccount user's passsha256). scadaproj is a shared monorepo — stage only the explicit glauth paths.
• Encrypted secrets store (gateway default): appsettings.json ships ${secret:ldap/mxgateway/bind}, resolved from the local encrypted store (seed with secret set ldap/mxgateway/bind <value>).
• Deployed hosts: env var MxGateway__Ldap__ServiceAccountPassword in the NSSM service environment.
• Dev boxes: dotnet user-secrets set "MxGateway:Ldap:ServiceAccountPassword" <value> (the server carries <UserSecretsId>mxaccessgw-server</UserSecretsId>).

See docs/GatewayConfiguration.md (the ServiceAccountPassword row) and glauth.md.

Preconditions

• SSH access to the GLAuth docker host 10.100.0.35 and to the deployed gateway host(s).
• Write access to scadaproj/infra/glauth/.
• Know which deployed hosts run LDAP-backed dashboard login:
  • 10.100.0.48 (windev) — primary; verify here.
  • wonder-app-vd03 — its dashboard is disabled. Check MxGateway:Ldap:Enabled there first. If LDAP is disabled (Enabled=false), it has nothing to bind and needs no env var — skip it.
• A generated replacement secret (see step 1). Generate the passsha256 per glauth.md ("Generate passsha256 from a plaintext password").

Cutover order

Follow this order so no window opens where the deployed dashboard cannot bind. Do not rotate GLAuth before the deployed hosts already carry the new value.

1. Generate the new secret in scadaproj/infra/glauth/. Pick a new password, compute its passsha256, and stage the change to the serviceaccount user in config.toml (do not docker compose up yet — the directory must keep honoring the OLD value until the deployed hosts carry the NEW one).

2. Pre-stage the NEW value on every LDAP-enabled deployed host via the env-var channel, so the host is ready the instant GLAuth flips:

   nssm get MxAccessGw AppEnvironmentExtra
   nssm set MxAccessGw AppEnvironmentExtra MxGateway__Ldap__ServiceAccountPassword=<new-value>
   # restart the service so the new environment is picked up
   nssm restart MxAccessGw
   

   Do this on 10.100.0.48, and on wonder-app-vd03 only if MxGateway:Ldap:Enabled=true there. (Alternatively seed the encrypted store with secret set ldap/mxgateway/bind <new-value>; the env var overrides the store and is the simplest per-host mechanism.) At this moment the deployed host holds the NEW value but GLAuth still honors the OLD one — binds still fail closed against the old directory, which is expected and brief; proceed immediately.

3. Rotate GLAuth on 10.100.0.35 to honor the new value:

   ssh 10.100.0.35
   cd ~/Desktop/scadaproj/infra/glauth
   docker compose up -d --force-recreate
   docker compose logs -f          # confirm clean startup, no TOML parse error
   

4. Verify dashboard login on the deployed host(s). Browse to the gateway dashboard on 10.100.0.48 and log in as multi-role / password (Administrator) — a successful login proves the search bind used the new service-account credential end-to-end. If wonder-app-vd03 runs LDAP, verify it too; if its dashboard/LDAP is disabled, no check is needed.

5. The repo change is already landed (removal of the committed value, <UserSecretsId>, the validator message naming the two channels, and doc/scrub updates). Nothing more to commit for the cutover.

6. Developers set user-secrets on next pull. After pulling, a dev box with no secret configured will fail startup with a validation message naming the exact command. One-time per machine:

   dotnet user-secrets set "MxGateway:Ldap:ServiceAccountPassword" <new-value>
   

   (value from scadaproj/infra/glauth/, never from a repo file).

Verifying the rotation

• Primary: dashboard /login as multi-role on 10.100.0.48 succeeds (step 4).
• wonder-app-vd03: only if MxGateway:Ldap:Enabled=true; otherwise no action.
• Live-LDAP integration tests (opt-in, only where the GLAuth instance is reachable):

  $env:MXGATEWAY_RUN_LIVE_LDAP_TESTS = "1"
  $env:MxGateway__Ldap__ServiceAccountPassword = "<new-value>"   # shell env only, never committed
  dotnet test src/ZB.MOM.WW.MxGateway.IntegrationTests/ZB.MOM.WW.MxGateway.IntegrationTests.csproj `
    --filter FullyQualifiedName~DashboardLdapLiveTests
  

  A green DashboardLdapLiveTests run confirms the new credential binds and searches. Where GLAuth is unreachable, document the suite as skipped per the docs/GatewayTesting.md opt-in matrix.
• Old value is dead: after step 3, a bind with the old password must fail. Do not test this from a shared-NAT box — GLAuth's 3-fail / 10-minute per-IP lockout can lock the whole office.

Rollback

If dashboard login breaks after step 3, restore the previous passsha256 in scadaproj/infra/glauth/config.toml, docker compose up -d --force-recreate, and re-point the deployed hosts' env var / store back to the previous value. Because the deployed hosts were pre-staged in step 2, the exposure window is only steps 2→4.

Done criteria

• GLAuth on 10.100.0.35 honors only the new value.
• Every LDAP-enabled deployed host binds with the new value (dashboard login verified).
• The source of truth scadaproj/infra/glauth/config.toml carries the new passsha256.
• No repo file (this one included) contains the old or new value.
• The SEC-36 tracker rows are Done with this runbook cited for the operator action.