mxaccessgw/docs/plans/2026-08-07-live-actions-sec36-tst30-publish.md

Live Actions: SEC-36 Rotation, TST-30 Second Runner, Client Publish — Implementation Plan

For Claude: REQUIRED SUB-SKILL: Use superpowers-extended-cc:executing-plans to implement this plan task-by-task (or subagent-driven-development in-session).

Goal: Execute the three repo-complete-but-live-pending operator actions: rotate the dev GLAuth service-account credential (SEC-36), register a second Gitea Actions runner (TST-30), and publish the five client packages at 0.2.0 (Java 0.2.1).

Architecture: Three independent workstreams executed by subagents. SEC-36 is a strictly ordered cutover (pre-stage hosts → flip GLAuth → verify → finalize) with secret-hygiene rules. TST-30 is infra work on docker host 10.100.0.35 plus a concurrency verification. Publish runs the existing guarded pack-clients.ps1 -Publish + tag-go-module.ps1 locally on macOS.

Tech Stack: ssh (BatchMode works to 10.100.0.35 and 10.100.0.48), PowerShell/nssm on windev, docker compose on 10.100.0.35, Gitea API (~/.zshenv has admin-scoped GITEA_USERNAME/GITEA_TOKEN), pwsh 7 on macOS.

---

Preflight facts (verified 2026-08-07 from this macOS box)

• ssh 10.100.0.35 OK. GLAuth container is zb-shared-glauth, compose working dir /home/dohertj2/zb-glauth (NOT the runbook's ~/Desktop/scadaproj/infra/glauth — that path does not exist on the host; the runbook must be corrected in Task 5). Runner container gitea-runner, compose working dir /opt/gitea.
• ssh 10.100.0.48 (windev) OK; powershell -NoProfile works; nssm at C:\Users\dohertj2\AppData\Local\Microsoft\WinGet\Links\nssm.exe.
• wonder-app-vd03 does NOT resolve from macOS — check it from windev (Task 2).
• Gitea API: token valid (/api/v1/user → 200), admin (/api/v1/admin/users → 200, POST /api/v1/admin/actions/runners/registration-token → 200).
• Local ~/Desktop/scadaproj/infra/glauth/config.toml exists (14 passsha256 entries) — the git source of truth.
• pwsh at /usr/local/bin/pwsh.

Secret hygiene (SEC-36, binding for every task)

• The new plaintext password lives ONLY in $SECRET_FILE = /private/tmp/claude-501/-Users-dohertj2-Desktop-MxAccessGateway/67849767-a07c-4afa-94e2-3ce4a39d8d23/scratchpad/sec36-new-secret (chmod 600), created in Task 1 and shredded in Task 5.
• Never echo/cat the plaintext to stdout, never put it in a commit, a repo file, a log line, or a command whose text is captured verbatim. Always load it into a shell variable from the file (val=$(cat "$SECRET_FILE")) and pass it via stdin or remote-side expansion, never inline in an ssh "...literal..." string where avoidable.
• The passsha256 hash MAY appear in config.toml commits — that is the established pattern (14 existing entries).
• The OLD password must never be printed either. Its only uses are: GLAuth keeps honoring it until Task 4, and the single old-bind-must-fail probe in Task 4.

---

Task 1: SEC-36 — Generate secret, stage GLAuth config change (repo + host copy diff)

Classification: high-risk Estimated implement time: ~5 min Parallelizable with: Task 2, Task 6, Task 9

Files:

• Modify: ~/Desktop/scadaproj/infra/glauth/config.toml (the serviceaccount user's passsha256) — DO NOT commit yet (Task 5 commits)
• Create: $SECRET_FILE (scratchpad, chmod 600)

Step 1: Generate the new secret and its hash

SECRET_FILE="/private/tmp/claude-501/-Users-dohertj2-Desktop-MxAccessGateway/67849767-a07c-4afa-94e2-3ce4a39d8d23/scratchpad/sec36-new-secret"
umask 077
openssl rand -base64 24 | tr -d '\n' > "$SECRET_FILE"
chmod 600 "$SECRET_FILE"
NEW_SHA=$(cat "$SECRET_FILE" | tr -d '\n' | shasum -a 256 | awk '{print $1}')
echo "$NEW_SHA"   # hash only — safe to display

Cross-check the hash recipe against glauth.md ("Generate passsha256 from a plaintext password") in this repo and follow that recipe if it differs.

Step 2: Diff host deployment config vs repo source of truth

ssh 10.100.0.35 'cat /home/dohertj2/zb-glauth/config.toml' > /tmp/host-glauth-config.toml 2>/dev/null || true
diff ~/Desktop/scadaproj/infra/glauth/config.toml /tmp/host-glauth-config.toml

Small drift (comments, ports) is fine — note it. If the serviceaccount stanza differs structurally, STOP and surface before editing.

Step 3: Edit the repo source of truth

In ~/Desktop/scadaproj/infra/glauth/config.toml, replace the passsha256 value of the [[users]] entry whose name/cn is serviceaccount with $NEW_SHA. Edit ONLY that line. Do not docker compose up anything yet.

Step 4: Record findings

Report: hash staged (show hash, never plaintext), drift summary from step 2, and confirm $SECRET_FILE exists with mode 600.

---

Task 2: SEC-36 — Determine wonder-app-vd03 LDAP status (via windev)

Classification: small Estimated implement time: ~3 min Parallelizable with: Task 1, Task 6, Task 9

Step 1: Try to reach vd03 from windev

ssh 10.100.0.48 'powershell -NoProfile -Command "Test-Connection wonder-app-vd03 -Count 1 -Quiet"'

Step 2: If reachable, read its gateway config for MxGateway:Ldap:Enabled

Try (in order, stop at first success): ssh hop from windev; reading \\wonder-app-vd03\c$\... appsettings/environment via PowerShell remoting (Invoke-Command -ComputerName wonder-app-vd03); or nssm get MxAccessGw AppEnvironmentExtra remotely. Look for MxGateway__Ldap__Enabled / appsettings Ldap:Enabled.

Step 3: Decide and record

• Enabled=false or host unreachable/no gateway service → vd03 is OUT of scope; record why (runbook says its dashboard is disabled — false is the expected answer).
• Enabled=true → vd03 is IN scope for Task 3 pre-staging; record the connection method that worked.

---

Task 3: SEC-36 — Pre-stage the NEW value on LDAP-enabled deployed hosts

Classification: high-risk Estimated implement time: ~4 min Parallelizable with: none (blocked by Tasks 1, 2)

Step 1: Pre-stage windev (10.100.0.48)

Load the secret locally, then set the env var remotely without leaking it into logged command text more than unavoidable (ssh arguments are not logged remotely by default; do NOT echo the value):

SECRET_FILE="/private/tmp/claude-501/-Users-dohertj2-Desktop-MxAccessGateway/67849767-a07c-4afa-94e2-3ce4a39d8d23/scratchpad/sec36-new-secret"
val=$(cat "$SECRET_FILE")
ssh 10.100.0.48 'powershell -NoProfile -Command "$v = [Console]::In.ReadLine(); $cur = (& nssm get MxAccessGw AppEnvironmentExtra) -join \"`n\"; Write-Output (\"CURRENT: \" + ($cur -replace \"Password=.*\", \"Password=<redacted>\")); & nssm set MxAccessGw AppEnvironmentExtra (\"MxGateway__Ldap__ServiceAccountPassword=\" + $v)"' <<< "$val"

CAUTION: nssm set AppEnvironmentExtra REPLACES the whole extra-environment block. First inspect nssm get MxAccessGw AppEnvironmentExtra (redacting any Password= values); if other variables exist, preserve them in the new value (newline-separated). Adapt quoting as needed — verify with a redacted nssm get afterwards.

Step 2: Restart the service

ssh 10.100.0.48 'nssm restart MxAccessGw'

Expected: service restarts. Binds against GLAuth now fail (old directory, new client value) — expected and brief; proceed immediately to Task 4.

Step 3: vd03 (only if Task 2 said IN scope) — same pre-stage + restart via the method Task 2 found.

---

Task 4: SEC-36 — Rotate GLAuth and verify end-to-end

Classification: high-risk Estimated implement time: ~5 min Parallelizable with: none (blocked by Task 3)

Step 1: Back up current host config, sync the staged config, recreate

ssh 10.100.0.35 'cp /home/dohertj2/zb-glauth/config.toml /home/dohertj2/zb-glauth/config.toml.bak-sec36'
scp ~/Desktop/scadaproj/infra/glauth/config.toml 10.100.0.35:/home/dohertj2/zb-glauth/config.toml

If Task 1's diff showed host-vs-repo drift beyond the serviceaccount line: do NOT wholesale-copy — instead edit only the serviceaccount passsha256 line in the host copy (sed on the host), so unrelated host-local drift is preserved.

ssh 10.100.0.35 'cd /home/dohertj2/zb-glauth && docker compose up -d --force-recreate && sleep 3 && docker compose logs --tail 30'

Expected: clean startup, no TOML parse error. On parse error: restore .bak-sec36, recreate, STOP, surface.

Step 2: Verify new credential binds (from the glauth host, ldapsearch or python)

SECRET_FILE=".../sec36-new-secret"   # full scratchpad path
val=$(cat "$SECRET_FILE")
ssh 10.100.0.35 'ldapsearch -x -H ldap://localhost:3893 -D "cn=serviceaccount,dc=zb,dc=local" -w "$(cat -)" -b "dc=zb,dc=local" "(cn=multi-role)" cn' <<< "$val"

Expected: search returns the multi-role entry. (If ldapsearch is missing on the host, run the equivalent from macOS against 10.100.0.35:3893, or use docker exec.) Adjust the bind DN to match the actual serviceaccount DN in config.toml.

Step 3: Verify the OLD value is dead — exactly ONE probe, from 10.100.0.35 itself

One deliberately failing bind with the old password must return invalid credentials. Only one attempt (3-fail/10-min per-IP lockout; never probe from a shared-NAT box). The old value: recover it transiently from config.toml.bak-sec36's hash? No — hash is not the plaintext. Instead: skip the plaintext probe if the old plaintext is not already known out-of-band; the hash replacement in config.toml is itself proof GLAuth no longer honors the old value (GLAuth compares against passsha256 only). Record that reasoning instead of probing blind.

Step 4: Verify dashboard login end-to-end on windev

curl -sk -o /dev/null -w '%{http_code}' -c /tmp/mxgw-cookies.txt https://10.100.0.48:5001/login

Find the actual dashboard port from windev config first (nssm get/appsettings; likely https). Then POST the login form as multi-role/password (the GLAuth TEST USER password, not the service account) and expect a redirect + __Host-MxGatewayDashboard (or MxGatewayDashboard) cookie:

curl -sk -o /dev/null -w '%{http_code}\n' -b /tmp/mxgw-cookies.txt -c /tmp/mxgw-cookies.txt -d 'username=multi-role&password=password' <dashboard-base>/login
grep -i mxgatewaydashboard /tmp/mxgw-cookies.txt

Inspect the login page HTML first for real form field names / antiforgery token; adapt. A successful multi-role login proves the service-account search bind works with the new credential end-to-end. If HTTP verification proves impractical (antiforgery), fall back to grepping the gateway log on windev for a successful LDAP bind/login line after attempting — or run the live-LDAP integration test from macOS:

export MXGATEWAY_RUN_LIVE_LDAP_TESTS=1
export MxGateway__Ldap__ServiceAccountPassword="$(cat "$SECRET_FILE")"
dotnet test src/ZB.MOM.WW.MxGateway.IntegrationTests/ZB.MOM.WW.MxGateway.IntegrationTests.csproj --filter FullyQualifiedName~DashboardLdapLiveTests

Expected: green. (This binds from macOS to 10.100.0.35:3893 directly — it verifies the credential, and the curl/log check verifies windev.)

Step 5: Rollback (only on failure) — restore .bak-sec36 on the host, docker compose up -d --force-recreate, re-point windev's env var back (old value from where it was before — if unknown, STOP and surface), nssm restart MxAccessGw.

---

Task 5: SEC-36 — Finalize: commit source of truth, dev secrets, runbook fix, tracker, cleanup

Classification: standard Estimated implement time: ~5 min Parallelizable with: none (blocked by Task 4)

Step 1: Commit and push the scadaproj glauth change (glauth paths ONLY)

cd ~/Desktop/scadaproj
git add infra/glauth/config.toml
git commit -m "sec(glauth): rotate serviceaccount passsha256 (mxaccessgw SEC-36)"
git push

(scadaproj is a shared monorepo — stage only this path. If the worktree has unrelated staged changes, use git commit -- infra/glauth/config.toml style isolation.)

Step 2: Set dev user-secrets on this macOS box

cd ~/Desktop/MxAccessGateway
cat "$SECRET_FILE" | tr -d '\n' | dotnet user-secrets set "MxGateway:Ldap:ServiceAccountPassword" --project src/ZB.MOM.WW.MxGateway.Server/ZB.MOM.WW.MxGateway.Server.csproj

(Check dotnet user-secrets set -h for stdin support; if unsupported, pass via "$(cat "$SECRET_FILE")" — acceptable, it's a local process arg.)

Step 3: Correct the runbook + flip tracker rows (mxaccessgw repo)

• docs/runbooks/SEC-36-ldap-credential-rotation.md: fix the host deployment path (/home/dohertj2/zb-glauth, container zb-shared-glauth; repo source of truth remains scadaproj/infra/glauth/), and note vd03's actual status per Task 2.
• Grep archreview/2026-07-12/remediation/ for SEC-36 pending-operator rows; flip to Done citing the runbook + today's date.

cd ~/Desktop/MxAccessGateway
grep -rn "SEC-36" archreview/2026-07-12/remediation/ docs/ | grep -iv binary
# edit the rows, then:
git add -A docs archreview && git commit -m "docs(sec-36): record live rotation done; correct runbook host paths"

Step 4: Shred the secret file

rm -P "$SECRET_FILE" 2>/dev/null || rm "$SECRET_FILE"

Step 5: Done-criteria check — walk the runbook's Done criteria list; report each as met/not-met.

---

Task 6: TST-30 — Recon existing runner config on 10.100.0.35

Classification: small Estimated implement time: ~4 min Parallelizable with: Task 1, Task 2, Task 9

Step 1: Inspect the existing runner

ssh 10.100.0.35 'cat /opt/gitea/docker-compose.yml 2>/dev/null || sudo cat /opt/gitea/docker-compose.yml; ls /opt/gitea'
ssh 10.100.0.35 'docker inspect gitea-runner --format "{{json .Mounts}}"; docker exec gitea-runner cat /config.yaml 2>/dev/null || true'

Find: image/version, config file location (look for container.network: traefik and capacity/maxParallel), data volume, registration state file, docker socket mount, labels.

Step 2: Check host capacity

ssh 10.100.0.35 'nproc; free -h; df -h / | tail -1'

Step 3: Decide (a)-variant — second container vs raising capacity on the existing runner. Runbook prefers a second instance; if the existing runner's config shows a simple capacity: 1 and resources are tight, raising capacity is the smaller change — but a second registered instance is the runbook default and survives one-runner wedge. Record the chosen variant, the exact compose/config snippets to reuse, and where the registration token goes.

---

Task 7: TST-30 — Register and start the second runner

Classification: high-risk Estimated implement time: ~5 min Parallelizable with: none (blocked by Task 6)

Step 1: Mint an instance-level registration token

source ~/.zshenv
curl -s -X POST -u "$GITEA_USERNAME:$GITEA_TOKEN" 'https://gitea.dohertylan.com/api/v1/admin/actions/runners/registration-token'

(Returns {"token": "..."} — a registration token, not a secret credential of lasting value; still avoid committing it.)

Step 2: Create the second runner instance per Task 6's plan

E.g. add a gitea-runner-2 service to the compose (distinct name + data volume, same image, same container.network: traefik, same socket mount), inject the token via the runner's registration env (GITEA_RUNNER_REGISTRATION_TOKEN) or act_runner register --no-interactive, then docker compose up -d gitea-runner-2 from /opt/gitea. Back up the compose file first (cp docker-compose.yml docker-compose.yml.bak-tst30). Do NOT touch the existing gitea-runner service definition.

Step 3: Confirm both runners online

curl -s -u "$GITEA_USERNAME:$GITEA_TOKEN" 'https://gitea.dohertylan.com/api/v1/admin/actions/runners' | python3 -m json.tool

Expected: ≥2 runners, both online. Also check docker logs of the new container for a clean registration + poll loop.

---

Task 8: TST-30 — Verify concurrency, gitea:3000 resolution, tracker

Classification: standard Estimated implement time: ~5 min Parallelizable with: none (blocked by Task 7)

Step 1: Trigger two concurrent runs

Push two scratch branches to mxaccessgw back-to-back (empty commits off main, branch names scratch/tst30-a, scratch/tst30-b):

cd ~/Desktop/MxAccessGateway
git push origin main:refs/heads/scratch/tst30-a
git commit --allow-empty -m "tst30 concurrency probe" && git push origin HEAD:refs/heads/scratch/tst30-b && git reset --hard HEAD~1

(Adapt: any two pushes that fan out jobs. Clean up branches after: git push origin :scratch/tst30-a :scratch/tst30-b.)

Step 2: Confirm parallel execution

Poll the runs API/UI: the second run's jobs must START before the first run finishes.

curl -s -u "$GITEA_USERNAME:$GITEA_TOKEN" 'https://gitea.dohertylan.com/api/v1/repos/dohertj2/mxaccessgw/actions/tasks' | python3 -m json.tool | head -60

Step 3: Confirm gitea:3000 resolves on the new runner — verify a job scheduled on runner-2 succeeds at checkout (checkout hits gitea:3000 over the traefik network); identify which runner took each job from the runs UI/API or runner logs.

Step 4: Flip TST-30 tracker rows in archreview/2026-07-12/remediation/ (grep TST-30) to Done with today's date; confirm docs/GatewayTesting.md prose is still accurate (it should be — it already describes the bypass as valid regardless of runner count). Commit.

---

Task 9: Publish — Preflight audit (versions, registry collisions, toolchains)

Classification: small Estimated implement time: ~4 min Parallelizable with: Task 1, Task 2, Task 6

Step 1: Audit source versions

cd ~/Desktop/MxAccessGateway
grep -n 'version' clients/rust/Cargo.toml | head -5
grep -n 'version' clients/python/pyproject.toml clients/python/src/zb_mom_ww_mxgateway/version.py
grep -n 'ClientVersion' clients/go/mxgateway/version.go
grep -n '<Version>' clients/dotnet/ZB.MOM.WW.MxGateway.Client/ZB.MOM.WW.MxGateway.Client.csproj src/ZB.MOM.WW.MxGateway.Contracts/ZB.MOM.WW.MxGateway.Contracts.csproj
grep -n 'version' clients/java/build.gradle | head -5
grep -rn 'CLIENT_VERSION' clients/java --include=*.java | grep -i mxgatewayclientversion

Expected: Rust/Python/Go/.NET/Contracts = 0.2.0; Java build.gradle AND MxGatewayClientVersion.CLIENT_VERSION = 0.2.1. Any mismatch → STOP, surface (do not bump versions yourself; that's a scope change).

Step 2: Query live registry for collisions

source ~/.zshenv
for u in 'nuget/ZB.MOM.WW.MxGateway.Client/0.2.0' 'nuget/ZB.MOM.WW.MxGateway.Contracts/0.2.0' 'pypi/zb-mom-ww-mxaccess-gateway-client/0.2.0' 'cargo/zb-mom-ww-mxgateway-client/0.2.0' 'maven/com.zb.mom.ww.mxgateway-zb-mom-ww-mxgateway-client/0.2.1'; do
  echo "$u => $(curl -s -o /dev/null -w '%{http_code}' -u "$GITEA_USERNAME:$GITEA_TOKEN" "https://gitea.dohertylan.com/api/v1/packages/dohertj2/$u")"
done

Expected: 404 for every target (unclaimed). Check the exact maven path convention against pack-clients.ps1's own guard code and use its convention. 200 anywhere → STOP, surface.

Step 3: Toolchain + workspace check

git -C ~/Desktop/MxAccessGateway status --porcelain   # must be clean (publish from a clean tree at origin/main)
for t in dotnet cargo go python3 gradle pwsh; do which $t; done

Also confirm clients/go module tag clients/go/v0.2.0 does NOT already exist: git ls-remote --tags origin 'clients/go/v*'.

---

Task 10: Publish — Run the guarded pack-and-publish

Classification: high-risk Estimated implement time: ~5 min dispatch (script runtime longer) Parallelizable with: none (blocked by Task 9)

Step 1: Run pack-clients with publish

cd ~/Desktop/MxAccessGateway
source ~/.zshenv
pwsh -NoProfile -File scripts/pack-clients.ps1 -Publish 2>&1 | tee /private/tmp/claude-501/-Users-dohertj2-Desktop-MxAccessGateway/67849767-a07c-4afa-94e2-3ce4a39d8d23/scratchpad/pack-clients-publish.log

Expected: per-language build+test+pack, collision guard prints "safe to publish" per artifact, uploads succeed. Timeout generously (Bash timeout 600000). If any language fails MID-loop, record exactly which artifacts pushed and which didn't — partial publish is the known failure mode; do not re-run blindly (re-run is safe only because the guard skips? NO — the guard ABORTS on existing versions. A re-run after partial publish will abort on the already-pushed artifact. If that happens, surface with the log; per-language -Languages selective re-run is the fix). If macOS cannot build a language (e.g. gradle/java env), use -Languages to publish what builds and surface the remainder — do not fake success.

Step 2: Verify each artifact now exists (200) — re-run Task 9 step 2's loop; expected 200 everywhere published.

---

Task 11: Publish — Go module tag

Classification: small Estimated implement time: ~3 min Parallelizable with: Task 10 (blocked by Task 9)

Step 1: Tag via the guarded script

cd ~/Desktop/MxAccessGateway
pwsh -NoProfile -File scripts/tag-go-module.ps1 -Version 0.2.0

Read the script's param block first (-Version name may differ; it validates semver and that version.go matches, then creates+pushes clients/go/v0.2.0). Expected: tag created and pushed to origin.

Step 2: Verify

git ls-remote --tags origin 'clients/go/v0.2.0*'

Expected: exactly one tag. Optionally GOPROXY=direct go list -m gitea.dohertylan.com/dohertj2/mxaccessgw/clients/go@v0.2.0 from a temp dir.

---

Task 12: Publish — Docs/tracker closeout

Classification: small Estimated implement time: ~4 min Parallelizable with: none (blocked by Tasks 10, 11)

Step 1: Update docs/ClientPackaging.md's versioning narrative if it claims 0.2.0/0.2.1 are unpublished (it currently records the maven 0.2.1 exception; add a dated line that 0.2.0 (Java 0.2.1) published on 2026-08-07). Grep archreview/2026-07-12/remediation/ for publish/CLI-39 pending-operator rows and flip to Done.

Step 2: Commit:

cd ~/Desktop/MxAccessGateway
git add docs archreview && git commit -m "docs(clients): record 0.2.0/0.2.1 publish + close operator actions"

Step 3: Report the full publish matrix (artifact → version → registry HTTP status).

---

Dependency graph

{T1, T2} ──▶ T3 ──▶ T4 ──▶ T5        (SEC-36, strictly serial after recon)
 T6 ──▶ T7 ──▶ T8                     (TST-30)
 T9 ──▶ {T10, T11} ──▶ T12            (Publish)

The three streams are mutually independent and run concurrently. All subagents run with model=opus per operator instruction.

Out of scope (explicitly)

• Option (b)/(c) runner topologies and the concurrency: ci.yml experiment (TST-30 runbook marks them escalation/optional).
• The five next-cycle candidate findings in archreview/2026-07-12/remediation/90-candidate-findings-next-cycle.md.
• Any client version bumps (versions are already landed; a mismatch is a STOP-and-surface).