mxaccessgw/docs/plans/2026-08-07-followups-windev-ldapfixtures-runners.md

Follow-Ups: windev Redeploy, LDAP Test Fixtures, Runner Hygiene — Implementation Plan

For Claude: REQUIRED SUB-SKILL: Use superpowers-extended-cc:subagent-driven-development (opus implementers; controller verifies ops evidence; final review pass).

Goal: Close the five items surfaced by the 2026-08-07 live-actions cycle: repair windev's crash-looping gateway and finish the deferred SEC-36 dashboard verification (NEXT-07), fix the DashboardLdapLiveTests fixture drift (NEXT-06), resolve the unexpected macOS instance runner, harden runner-1's plaintext registration token, and verify the cargo Bearer fix.

Architecture: Three independent live streams (windev serial: 1→2→3; repo test fix: 4; Gitea/runner hygiene: 5, 6) run concurrently; task 7 closes out docs/trackers. No contract, gateway-logic, or client changes — one test-file edit (Task 4) plus live ops plus docs.

Tech Stack: SSH + PowerShell -EncodedCommand (windev 10.100.0.48), SSH + docker compose (10.100.0.35), Gitea admin API (gitea.dohertylan.com, token via ~/.zshenv GITEA_TOKEN), NSSM, xUnit live-LDAP suite, GLAuth at 10.100.0.35:3893.

---

Preflight facts (verified before planning)

• Unpushed local mxaccessgw commits: 0566716, 9760497, 5b153da, 41e8648 (all docs-only). origin/main = a346d51 — contains all current code, so windev can build from origin/main without any push.
• windev (10.100.0.48): NSSM service MxAccessGw; deployed Server build of 2026-06-25 (Auth 0.1.2.0, supports auth-DB schema 2) crash-loops on C:\ProgramData\MxGateway\gateway-auth.db migrated to schema 3 on 2026-07-15 (AuthStoreMigrationException, ~3.8k–10k Hosting-failed events/day). The NEW LDAP secret is already staged as the 10th AppEnvironmentExtra entry (SEC-36 Task 3) — preserve it.
• DashboardLdapLiveTests.cs (src/ZB.MOM.WW.MxGateway.IntegrationTests/): uses admin/admin123 (3 tests) and readonly/readonly123. Directory reality (scadaproj/infra/glauth/config.toml): admin exists, password is the standard dev test password (password, hash 5e884898…42d8 — same as multi-role), and IS in GwAdmin (othergroups [5610, 5701]); readonly does not exist; gw-viewer (primarygroup 5611 = GwReader, NOT GwAdmin) is the natural not-an-admin fixture. Test binds MxGateway:Ldap from appsettings.json (Server: localhost) + env overrides — so the live run needs MxGateway__Ldap__Server=10.100.0.35 as well as MxGateway__Ldap__ServiceAccountPassword (from Mac user-secrets, never printed).
• Gitea instance runners (GET /api/v1/admin/actions/runners): id 1 gitea-runner (cap 4), id 4 macos-local-Josephs-MacBook-Pro (unexpected, online, labels overlap ubuntu-latest), id 5 gitea-runner-2 (cap 2).
• 10.100.0.35:/opt/gitea/docker-compose.yml (+ docker-compose.yml.bak-tst30): runner-1's registration token inline in plaintext env, file world-readable. runner-2 uses GITEA_RUNNER_REGISTRATION_TOKEN_FILE: /run/secrets/runner_token ← 0600 /opt/gitea/runner_token. runner-1 data volume /opt/gitea/runner:/data (its .runner credential persists — the registration env is only needed for first registration).
• Cargo Bearer fix already applied (~/.zshenv, backup ~/.zshenv.bak-cli39) and documented (docs/ClientPackaging.md, commit 5b153da). Task 7 verifies; no further action expected.

Secret hygiene (binding, all tasks)

• Never print the LDAP service-account password, GITEA_TOKEN, cargo token, runner registration tokens, or API keys — not in commands, logs, commits, or reports. Read the LDAP password from dotnet user-secrets list into an env var without echoing (e.g. export MxGateway__Ldap__ServiceAccountPassword="$(dotnet user-secrets list --project src/ZB.MOM.WW.MxGateway.Server | awk -F' = ' '/ServiceAccountPassword/ {print $2}')").
• Documented dev test users (multi-role/password, admin/password, gw-viewer/password) are NOT secrets — glauth.md publishes them; fine in code/commits.
• SSH→windev PowerShell: always powershell -NoProfile -EncodedCommand <base64-UTF16LE>; never put secrets inside EncodedCommand blobs or argv.

---

Task 1: NEXT-07 — Recon windev deployment layout + schema support

Classification: standard — read-only recon, but its output gates a service redeploy Estimated implement time: ~5 min Parallelizable with: Task 4, Task 5, Task 6

Files: none edited. SSH recon on 10.100.0.48 + repo/scadaproj reads on the Mac.

Determine everything Task 2 needs, and confirm the fresh-deploy path is safe:

1. nssm get MxAccessGw Application, AppDirectory, AppParameters, AppEnvironmentExtra (count entries; do NOT print values of secret-bearing entries — names only).
2. Inventory the deployed dir: path, ZB.MOM.WW.MxGateway.Server.exe timestamp, whether appsettings.json/appsettings.Production.json in the deploy dir differ from repo origin/main (diff; windev-specific config must survive the redeploy).
3. Confirm build feasibility on windev: dotnet --list-sdks (need 10.x), locate an existing mxaccessgw checkout/worktree (CI uses scripts/ci/windev-worker-ci.ps1 — find its worktree path) or pick a fresh clone location. Confirm git fetch reaches origin/main = a346d514dd24e775640e5667aa7cd8e561fec68a.
4. Confirm current code supports auth-DB schema 3: find the auth-store supported-schema constant (ZB.MOM.WW.Auth packages — check the package version the Server at origin/main references, and/or the migration code in the shared scadaproj libs) and state the evidence. If current code does NOT support schema 3, STOP — report, do not deploy.
5. Gateway endpoints for verification: bound URLs/ports (from deployed config/env), dashboard scheme (http vs https → cookie will be MxGatewayDashboard vs __Host-…).
6. Check what migrated the DB to schema 3 on 2026-07-15 (event log / file timestamps) — only to confirm schema 3 is the shared-lib current version, not an anomaly.

Step: report all findings as structured text (no secrets); no changes, no commits.

Task 2: NEXT-07 — Build current Server on windev and redeploy the service

Classification: high-risk — replaces a running (crash-looping) service's binaries Estimated implement time: ~10 min Parallelizable with: none (needs Task 1)

Files: none in repo. windev filesystem + NSSM only.

Using Task 1's facts:

1. On windev, fetch/checkout origin/main (a346d51…) in the build worktree/clone.
2. dotnet publish src/ZB.MOM.WW.MxGateway.Server -c Release (match deployed layout/RID from Task 1; framework-dependent vs self-contained must match what NSSM Application points at).
3. Stop the service (nssm stop MxAccessGw), confirm process exited.
4. Backup: deployed dir → sibling *.bak-next07 copy; copy C:\ProgramData\MxGateway\gateway-auth.db (+ -wal/-shm if present) to gateway-auth.db.bak-next07. Never delete the live DB.
5. Deploy publish output over the deploy dir, then restore any windev-specific config files identified in Task 1 (do not clobber live overrides; NSSM env entries are untouched by file copies but verify count unchanged after start).
6. nssm start MxAccessGw; verify: service state RUNNING and stable ≥60 s (no restart cycle), Application event log shows clean host start and zero new AuthStoreMigrationException / Hosting failed to start after the start timestamp, bound port answers (e.g. dashboard root or health endpoint returns HTTP).
7. Rollback if unhealthy: stop, restore *.bak-next07 dir, start, report.

Step: report deployed SHA, verification evidence, backup paths. No repo commits.

Task 3: SEC-36 deferred verification + NEXT-07/runbook closeout

Classification: standard Estimated implement time: ~6 min Parallelizable with: none (needs Task 2)

Files:

• Modify: docs/runbooks/SEC-36-ldap-credential-rotation.md (Correction 3 — mark the deferred dashboard check done, dated)
• Modify: archreview/2026-07-12/remediation/90-candidate-findings-next-cycle.md (NEXT-07 → resolved 2026-08-07, evidence one-liner)

1. Complete SEC-36 step 4 against the repaired windev gateway: log in to the dashboard as multi-role/password end-to-end. Preferred: curl flow — GET /login (capture antiforgery token + cookie), POST credentials, expect success redirect + auth cookie (name per Task 1 scheme). If the login page resists scripting (Blazor circuit), report exactly why and fall back to asserting a fresh DashboardLdapLiveTests green run (Task 4) plus windev log evidence of successful LDAP bind on a manual attempt.
2. Update the two docs; commit locally (docs(sec-36,next-07): …), do NOT push.

Task 4: NEXT-06 — Fix DashboardLdapLiveTests fixtures to match the shared directory

Classification: small — one test file, but must go green against live GLAuth Estimated implement time: ~6 min Parallelizable with: Task 1, Task 5, Task 6

Files:

• Modify: src/ZB.MOM.WW.MxGateway.IntegrationTests/DashboardLdapLiveTests.cs
• Modify: archreview/2026-07-12/remediation/90-candidate-findings-next-cycle.md (NEXT-06 → resolved)
• Possibly modify: docs/GatewayTesting.md (live-LDAP opt-in row: document the MxGateway__Ldap__Server override needed when GLAuth is not localhost)

1. Read DashboardAuthenticator first: confirm a user who binds successfully but maps to no role yields Succeeded == false (drives the gw-viewer fixture).
2. Fix fixtures: admin/admin123 → admin/password (positive + wrong-password + unreachable tests); readonly/readonly123 → gw-viewer/password (exercises user-binds-but-lacks-GwAdmin; keep the no-password-leak assertion, updating the asserted literal). Update XML doc comments to match. Keep MXAccess-repo style rules (TreatWarningsAsErrors).
3. Build: dotnet build src/ZB.MOM.WW.MxGateway.IntegrationTests (macOS OK — net10.0).
4. Live run (env only, never echo the password): MXGATEWAY_RUN_LIVE_LDAP_TESTS=1 MxGateway__Ldap__Server=10.100.0.35 MxGateway__Ldap__ServiceAccountPassword=<from user-secrets> dotnet test … --filter FullyQualifiedName~DashboardLdapLiveTests → expect 5/5 passed (this is also positive live proof of the SEC-36 service-account bind).
5. Update tracker row (+ GatewayTesting.md if the Server-override note is missing); commit locally (test(ldap): …), do NOT push.

Task 5: Resolve the unexpected macOS instance runner (id 4)

Classification: standard — evidence-gated removal of a live runner registration Estimated implement time: ~6 min Parallelizable with: Task 1, Task 4, Task 6

Files:

• Modify: docs/runbooks/TST-30-second-ci-runner.md (the Correction paragraph mentions "id 4 — an unrelated local macOS runner" — update to final state)

1. Evidence, local: pgrep -fl act_runner, launchctl list | grep -i act, brew services list | grep -i act, look for ~/.runner/act_runner config dirs. Evidence, Gitea (token from ~/.zshenv, never printed): runner detail for id 4 (labels, last online), and whether any recent runs' jobs report runner_id == 4 (GET /repos/{owner}/{repo}/actions/runs?… → …/runs/{id}/jobs for both mxaccessgw and lmxopcua recent runs).
2. Decision rule: the runner advertises ubuntu labels from a macOS host, so it can steal Linux container jobs → remove it unless evidence shows it deliberately serves jobs the docker runners cannot (none expected). Removal = stop the local act_runner process AND disable its autostart (launchd/brew), then DELETE /api/v1/admin/actions/runners/4. Keep the local config file (renamed *.disabled-2026-08-07) so re-registering with mac-specific labels stays easy; note the re-registration recipe in the runbook edit.
3. Verify: admin runner list shows only ids 1 and 5, both online; no act_runner process locally; a pgrep after 60 s still empty (nothing respawned).
4. Update the TST-30 runbook correction paragraph; commit locally, do NOT push.

Task 6: Harden runner-1's registration token on 10.100.0.35

Classification: standard — touches the live CI stack's compose file Estimated implement time: ~7 min Parallelizable with: Task 1, Task 4, Task 5

Files: none in repo (host /opt/gitea/ only; runbook note lands in Task 7 if needed).

1. Preconditions on the host: confirm runner-1's /data/.runner exists in its volume (registration credential persists → the registration env var is no longer needed); confirm both runners idle (no act_runner-spawned job containers, no in-progress runs via API) before recreating.
2. Edit /opt/gitea/docker-compose.yml (backup first → docker-compose.yml.bak-tst30b): replace runner-1's inline GITEA_RUNNER_REGISTRATION_TOKEN: <plaintext> with the same _FILE/secrets pattern runner-2 uses (/opt/gitea/runner_token, 0600). Do NOT touch the gitea service definition.
3. docker compose up -d --no-deps the runner-1 service only; verify it comes back online in the admin runner list and its .runner identity is unchanged (still id 1).
4. Tighten perms: chmod 600 /opt/gitea/docker-compose.yml docker-compose.yml.bak-tst30 docker-compose.yml.bak-tst30b (verify compose stack still operable by the deploy user).
5. Rotate the leaked registration token if the deployment allows: docker exec … gitea actions generate-runner-token (or admin API) — if Gitea offers no invalidation of the old value, say so explicitly in the report (residual risk: LAN actor could register a rogue runner until rotation) rather than claiming it rotated.
6. Verify CI still works: trigger nothing; just confirm both runners online and the token file perms; a real push lands naturally later. Report evidence.

Task 7: Closeout — cargo Bearer verification, docs/tracker sync, commits

Classification: small Estimated implement time: ~5 min Parallelizable with: none (needs Tasks 3, 4, 5, 6)

Files:

• Modify: archreview/2026-07-12/remediation/90-candidate-findings-next-cycle.md (final state of NEXT-06/NEXT-07 rows if Tasks 3/4 left anything)
• Possibly modify: docs/GatewayTesting.md / TST-30 runbook (runner topology now ids 1+5 only; token-hardening note)

1. Cargo Bearer verification (no printing): assert ~/.zshenv line matches CARGO_REGISTRIES_DOHERTJ2_GITEA_TOKEN="Bearer …" via grep -c, confirm docs/ClientPackaging.md note present (commit 5b153da); check no other credential location (CI secrets, windev profiles) publishes to cargo — expected none.
2. Sweep: every doc touched this cycle consistent (runbooks, trackers, GatewayTesting.md); git grep for stale phrases ("crash-loop… pending", "id 4", "admin123") and fix.
3. Commit remaining doc changes locally; do NOT push. List the full unpushed stack in the report.

---

Out of scope

• Pushing any mxaccessgw commits (user decides; stack listed at closeout).
• The five next-cycle candidate findings other than NEXT-06/NEXT-07.
• Auth-DB restore path for windev (fresh deploy chosen — preserves schema-3 data).
• ci.yml changes (labels, concurrency groups).

Dependency graph

{1} → 2 → 3 ┐
{4}          ├→ 7
{5}          │
{6} ─────────┘