mxaccessgw/archreview/remediation/60-testing-docs-gaps.md

# Testing, Documentation & Underdeveloped Areas — Remediation Design & Implementation

Source review: [60-testing-docs-gaps.md](../60-testing-docs-gaps.md) · Generated: 2026-07-09

This domain has no source-code defects to fix — its findings are process, coverage, documentation-currency, and half-shipped-feature gaps. The remediation therefore centres on standing up CI, filling the highest-value test gaps, finishing (or explicitly re-scoping) the session-resilience epic, and a focused doc-drift + repo-root triage pass. Every citation below was re-verified against the working tree on 2026-07-09; corrections are noted inline.

## Finding index

| ID | Sev | Title | Roadmap | Effort | Depends on | Files |
|----|-----|-------|---------|--------|-----------|-------|
| TST-01 | High | Reconnect/replay has no e2e test and no client consumer | P2 | L | TST-04 | oldtasks.md, clients/*, EventStreamServiceTests.cs |
| TST-02 | High | Reconnect owner re-validation not implemented | P0 | M | TST-04 | Sessions/SessionManager.cs, Grpc/EventStreamService.cs |
| TST-03 | High | No CI exists | P1 | M | — | (new) .gitea/workflows/ or .github/workflows/ |
| TST-04 | High | Session-resilience epic 16/28 tasks unfinished | P2 | L | — | oldtasks.md, docs/plans/2026-06-15-session-resilience.md.tasks.json |
| TST-05 | Medium | Real-worker control/COM paths verified opt-in only | P1 | S | TST-03 | WorkerLiveMxAccessSmokeTests.cs |
| TST-06 | Medium | Dashboard live-data path untested | — | M | — | Dashboard/DashboardLiveDataService.cs |
| TST-07 | Medium | Real-clock sleeps with negative assertions are latent flakes | — | S | — | WorkerClientTests.cs:431, SessionManagerTests.cs:400 |
| TST-08 | Medium | Full-suite orphaned testhost processes | P1 | M | — | Tests fixtures (pipe/hosted-service disposal) |
| TST-09 | Medium | Health checks cover only the auth store | — | M | — | GatewayApplication.cs:71 |
| TST-10 | Medium | Deployment/upgrade undocumented and hand-rolled | — | M | — | (new) docs/Deployment.md, scripts/ |
| TST-11 | Medium | No version discipline on server; client versions drift | P2 | M | — | src/Directory.Build.props, clients/* |
| TST-12 | Medium | CLAUDE.md misstates default retention behaviour | P0 | S | — | CLAUDE.md:79 |
| TST-13 | Medium | gateway.md carries stale design-era sketches | P2 | S | — | gateway.md:291,738,330 |
| TST-14 | Medium | Repo-root working artifacts need triage | P2 | S | — | *-docs-*.md, oldtasks.md, stillpending.md |
| TST-15 | Medium | Dashboard EventsHub has no per-session ACL | P2 | M | TST-04 | Dashboard/Hubs/EventsHub.cs:39 |
| TST-16 | Medium | `Dashboard:ShowTagValues` is a dead flag | — | S | — | Configuration/DashboardOptions.cs:62 |
| TST-17 | Medium | Vendor-gated alarm parity residuals silently lossy | — | S | — | Worker/MxAccess/WnWrapAlarmConsumer.cs:261 |
| TST-18 | Low | Hosted-service wrappers untested | — | S | — | SessionLeaseMonitorHostedService.cs, OrphanWorkerCleanupHostedService.cs |
| TST-19 | Low | Keep FakeWorkerHarness canned replies in lockstep | — | S | — | Tests/Gateway/Workers/Fakes/FakeWorkerHarness.cs |
| TST-20 | Low | E2E script papers over a real advise-without-consumer sharp edge | — | S | — | scripts/run-client-e2e-tests.ps1 |
| TST-21 | Low | Log rotation configured but minimal | — | S | — | Server/appsettings.json:8 |
| TST-22 | Low | Config-shape JSON block omits documented keys | — | S | — | docs/GatewayConfiguration.md:12 |
| TST-23 | Low | Bidirectional `Session` RPC never built | P2 | S | — | gateway.md:330 |
| TST-24 | Low | Client wire behaviour has no automated verification | — | M | TST-03 | clients/*, (new) in-process fake gateway |

---

## TST-01 — Reconnect/replay has no e2e test and no client consumer   `High` · `P2`

**Finding.** The server emits the `ReplayGap` sentinel and replays the ring on reconnect (unit-covered in `src/ZB.MOM.WW.MxGateway.Tests/Gateway/Grpc/EventStreamServiceTests.cs` and `MxAccessGatewayServiceTests.cs`), but epic Task 15 (fake-worker reconnect integration test) and Task 14 (client `ReplayGap` handling in all five clients) remain pending — verified `oldtasks.md:41-44` (`⏳ Task 14 (#121)`, `⏳ Task 15 (#122)`). No client parses the `ReplayGap` message.

**Impact.** The default-on reconnect protocol (`DetachGraceSeconds = 30`, `ReplayBufferCapacity = 1024` — both verified in code, see TST-12) is unproven end-to-end and unusable by every official client: a client that reconnects silently drops the gap marker and cannot tell replayed events from live ones, or that a gap was truncated.

**Design.** Land Tasks 14 and 15 as the completion of Phase 3 (co-designed with TST-02, TST-04). Two parts:
1. **Integration test (Task 15):** drive `FakeWorkerHarness` through advise → detach subscriber (keep session in grace) → emit events into the replay ring → reconnect a new `StreamEvents` with a `last_sequence` cursor → assert the exact replayed set plus a `ReplayGap` when the requested cursor predates the ring. This lives entirely in fakes; no live COM needed.
2. **Client consumers (Task 14):** each client's event-stream reader must surface `ReplayGap` as a typed, observable signal (not a dropped frame). Carry the per-language `optional`/presence-check idiom note from `oldtasks.md:44` — proto3 `optional` message fields need a `HasField`/`Option`/`is not None` presence check per language.

Rejected alternative: advertise reconnect as shipped and document the gap. Rejected because the roadmap already lists it as shipped and CLAUDE.md documents it as a feature (TST-12); leaving clients unable to consume it is a latent correctness trap, not a documentation nuance.

**Implementation.**
- Test: add `GatewayEndToEndReconnectReplayTests` under `src/ZB.MOM.WW.MxGateway.Tests/Gateway/` reusing `FakeWorkerHarness` and `ManualTimeProvider` to control the detach-grace clock.
- Clients: `clients/dotnet`, `clients/go`, `clients/rust`, `clients/python`, `clients/java` event readers — expose `ReplayGap` on the stream item type. Cross-reference the clients-domain report (`CLI`) for the shared stream-surface shape.
- Docs: `docs/Sessions.md` "Reconnect and replay" — change "server-side" caveat to "supported end-to-end"; each client README documents the gap signal.

**Verification.** `dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter FullyQualifiedName~GatewayEndToEndReconnectReplayTests`; per-client build/test per CLAUDE.md's client table (`gradle test`, `cargo test --workspace`, `python -m pytest`, `go test ./...`, `dotnet build clients/dotnet/...`).

---

## TST-02 — Reconnect owner re-validation not implemented   `High` · `P0`

**Finding.** Epic Task 13 ("Owner re-validation on reconnect") is pending — verified `oldtasks.md:40` (`⏳ Task 13 (#120): Owner re-validation on reconnect — blockedBy 12, 1`). Nothing ties a resuming `StreamEvents` call to the API key that opened the session beyond possessing the `event` scope and knowing the session id.

**Impact.** With fan-out (`AllowMultipleEventSubscribers`) or detach-grace enabled, any `event`-scoped key that learns a session id can attach to another key's session and receive its replayed and live data. This is a trust-boundary hole in a window that is **on by default** (`DetachGraceSeconds = 30`). It is a security control, not a resilience nicety.

**Design.** Persist the opening key's identity on the session and enforce it on every `StreamEvents` attach/reattach. Options:
- **(recommended)** Store the authenticated key id (already available on the auth context from `GatewayGrpcAuthorizationInterceptor`) on the session record at `OpenSession`; in `EventStreamService`/`MxAccessGatewayService.StreamEvents`, reject an attach whose caller key id ≠ the session owner with `PermissionDenied`. Admin-scope override deferred to Phase 4 (TST-15).
- Rejected: gate solely on the `event` scope (status quo) — that is exactly the hole.

This is a P0 item in the roadmap ("Close the reconnect owner re-validation gap or flip retention defaults off until it lands"). If Task 13 cannot land immediately, the safe interim is to flip `DetachGraceSeconds`/fan-out defaults off and correct CLAUDE.md (TST-12) to match — but the recommended path is to implement re-validation because the feature is already documented as shipped.

**Implementation.**
- `src/ZB.MOM.WW.MxGateway.Server/Sessions/SessionManager.cs` (session record gains an owner key id) and `Sessions/GatewaySession.cs`.
- `src/ZB.MOM.WW.MxGateway.Server/Gateway/Grpc/EventStreamService.cs` / `MxAccessGatewayService.cs` — owner check on attach.
- Tests: `EventStreamServiceTests` (owner match / mismatch / admin override), plus an attach-by-foreign-key rejection case in the reconnect integration test (TST-01).
- Docs: `docs/Sessions.md` and `docs/DesignDecisions.md` — document the owner-binding rule; CLAUDE.md Authentication section note that session attach is owner-scoped.

**Verification.** `dotnet test ... --filter FullyQualifiedName~EventStreamServiceTests` and the TST-01 integration filter; `dotnet build src/ZB.MOM.WW.MxGateway.Server`.

---

## TST-03 — No CI exists   `High` · `P1`

**Finding.** No pipeline files anywhere — verified: `.github/` and `.gitea/` absent, no `*.yml`/`Jenkinsfile`/`azure-pipelines.yml` at repo root. Every documented verification step is manual.

**Impact.** The five-language, dual-runtime (net10 / net48-x86) matrix is precisely the shape that breaks silently. Repo history shows the failure class is routine: net48 CS0246 on unregenerated protos, Java tracked-generated-file churn, seven-week-stale client descriptors (cross-domain `IPC`/`CLI`).

**Design.** Stand up a minimal pipeline. Because the x86 Worker only builds on a Windows host with MXAccess installed, split into a portable job (any runner) and a Windows job (self-hosted runner on windev, `10.100.0.48`). Live-MXAccess runs stay opt-in and scheduled, never gating a push.

Minimal pipeline (Gitea Actions syntax; `.github/workflows/` is drop-in equivalent if the mirror is GitHub):

```yaml
# .gitea/workflows/ci.yml
name: ci
on: [push, pull_request]
jobs:
  portable:                       # any linux/mac runner — no MXAccess, no x86
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Build NonWindows solution
        run: dotnet build src/ZB.MOM.WW.MxGateway.NonWindows.slnx -c Release
      - name: Gateway fake-worker tests
        run: dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj -c Release
      - name: Descriptor / codegen freshness
        run: pwsh scripts/check-descriptors.ps1   # see below; fail if regen would change tracked output
      - name: Client builds/tests
        run: |
          dotnet build clients/dotnet/ZB.MOM.WW.MxGateway.Client.slnx -c Release
          (cd clients/go && gofmt -l . && go build ./... && go test ./...)
          (cd clients/rust && cargo fmt --check && cargo test --workspace && cargo clippy --workspace --all-targets -- -D warnings)
          (cd clients/python && python -m pip install -e ".[dev]" && python -m pytest)
      - name: Java client
        run: (cd clients/java && gradle test)      # needs a JDK 17 runner; Mac dev box has none
  windows:                        # self-hosted windev runner — x86 worker + net48 tests
    runs-on: [self-hosted, windows]
    steps:
      - uses: actions/checkout@v4
      - run: dotnet build src/ZB.MOM.WW.MxGateway.Worker/ZB.MOM.WW.MxGateway.Worker.csproj -p:Platform=x86
      - run: dotnet test src/ZB.MOM.WW.MxGateway.Worker.Tests/ZB.MOM.WW.MxGateway.Worker.Tests.csproj -p:Platform=x86
  live-mxaccess:                  # scheduled only, never gates a push
    if: github.event_name == 'schedule'
    runs-on: [self-hosted, windows, mxaccess]
    env: { MXGATEWAY_RUN_LIVE_MXACCESS_TESTS: "1" }
    steps:
      - uses: actions/checkout@v4
      - run: dotnet test src/ZB.MOM.WW.MxGateway.IntegrationTests/ZB.MOM.WW.MxGateway.IntegrationTests.csproj
```

**Freshness check.** The `descriptors/codegen freshness` step is the highest-leverage single guard (it protects the Generated/-commit rule and the stale client descriptor set). Implement `scripts/check-descriptors.ps1` to: rebuild `ZB.MOM.WW.MxGateway.Contracts`, regenerate the published client descriptor set, then `git diff --exit-code` the tracked `Generated/` and `clients/proto/descriptors/` paths — non-empty diff fails the job. This addresses the `IPC` domain's stale-descriptor finding directly; coordinate the exact regen command with that report.

Note the two known-noisy interactions to encode as CI hygiene: net48 requires the regenerated `Generated/*.cs` to be **committed** (regen-then-commit), while the Java build **regenerates and dirties** `clients/java/src/main/generated/.../MxaccessGateway.java` with spurious version churn that must be reverted when no `.proto` changed. The Java job must `git checkout` that file after `gradle test` (or the freshness check must exclude it) so the build stays green.

**Implementation.** New `.gitea/workflows/ci.yml` (+ `.github/workflows/ci.yml` if mirrored); new `scripts/check-descriptors.ps1`. Register a self-hosted runner on windev with `windows`/`mxaccess` labels. Docs: add a "Continuous Integration" section to `docs/GatewayTesting.md` describing the jobs and the scheduled live cadence; note the Windows-runner requirement for the x86 worker.

**Verification.** Push a branch and confirm the `portable` and `windows` jobs go green; deliberately introduce a proto change without regen and confirm the freshness step fails.

---

## TST-04 — Session-resilience epic 16/28 tasks unfinished   `High` · `P2`

**Finding.** Verified `oldtasks.md:39-63`: pending are Task 13 (owner re-validation, TST-02), 14–15 (client ReplayGap + reconnect test, TST-01), 16–19 (per-session dashboard ACL, TST-15), and 20–28 (orphan-worker reattach, incl. the `EnableOrphanReattach` flag at Task 26 that does not yet exist). The repo advertises reconnect and fan-out as shipped while their trust boundary and client support are absent.

**Impact.** The default-on retention window has no owner check (security, TST-02); no client can consume the protocol (TST-01); Phase 5 reattach would reverse a documented CLAUDE.md invariant ("Gateway restart does not reattach orphan workers"). The epic is in a state where "shipped" and "planned" are entangled in the same docs.

**Design.** Treat the epic as three decisions, not one:
- **Phase 3 (Tasks 13–15): finish now, as one unit.** These make the already-shipped server behaviour real and safe — TST-01 + TST-02 are that work.
- **Phase 4 (Tasks 16–19): scope explicitly.** Per-session dashboard ACL is TST-15; decide the Viewer default (admin-sees-all vs strict per-session, the open decision at `oldtasks.md:52`) before implementing.
- **Phase 5 (Tasks 20–28): re-scope or defer explicitly.** Orphan-worker reattach reverses a CLAUDE.md invariant and adds a manifest store + phone-home protocol. Recommendation: mark Phase 5 as **deferred, not planned** unless a concrete requirement exists; do not leave it as implied backlog. The `EnableOrphanReattach` flag must not be referenced anywhere as if it exists until Task 26 lands.

This finding is the umbrella; TST-01/02/15 are its actionable slices. The remediation here is the governance action: update the epic's tracking source and the roadmap so "shipped" claims match reality.

**Implementation.**
- Update `docs/plans/2026-06-15-session-resilience.md.tasks.json` and its mirror `oldtasks.md` to reflect the Phase-3-finish / Phase-4-scope / Phase-5-defer decision (retire the mirror once the epic resumes or is closed, per TST-14).
- CLAUDE.md "Repository-Specific Conventions" — the reconnect/fan-out paragraph must state which parts are shipped-and-consumable vs server-only-pending-clients until TST-01 lands.

**Verification.** Documentation-only for the umbrella; the slices carry their own build/test verification (TST-01, TST-02, TST-15).

---

## TST-05 — Real-worker control/COM paths verified opt-in only   `Medium` · `P1`

**Finding.** All eleven late-added command kinds are unit-tested against fakes and live-verified once on the dev rig (`stillpending.md` §1.1), but the default suite exercises `Ping`/`GetWorkerInfo`/`DrainEvents`/`ShutdownWorker` only through `FakeWorkerHarness.RespondToControlCommandAsync` (verify current line range in `src/ZB.MOM.WW.MxGateway.Tests/Gateway/Workers/Fakes/FakeWorkerHarness.cs`), which returns canned replies.

**Impact.** A worker-side regression in these paths is invisible until someone sets `MXGATEWAY_RUN_LIVE_MXACCESS_TESTS=1`.

**Design.** Co-designed with TST-03: schedule `WorkerLiveMxAccessSmokeTests` on the windev runner on a cadence (the `live-mxaccess` job above). No new test code needed beyond ensuring the existing live suite covers the eleven command kinds; if any are absent, add live cases. This converts "opt-in, run by memory" into "runs nightly, reports failures."

**Implementation.** Covered by the `live-mxaccess` scheduled job in TST-03. Audit `WorkerLiveMxAccessSmokeTests.cs` for coverage of each command kind; add missing `[LiveMxAccessFact]` cases. Docs: `docs/GatewayTesting.md` opt-in matrix notes the scheduled cadence.

**Verification.** On windev: `MXGATEWAY_RUN_LIVE_MXACCESS_TESTS=1 dotnet test src/ZB.MOM.WW.MxGateway.IntegrationTests/...`.

---

## TST-06 — Dashboard live-data path untested   `Medium` · `—`

**Finding.** `Dashboard/DashboardLiveDataService.cs` (verified present; owns the shared lazily-opened gateway session backing `/browse` live values) has no test class; `EventsHub`/`AlarmsHubPublisher` hub methods are likewise untested.

**Impact.** The one dashboard component that holds a real worker session and can fault it has no regression net; a fault-recovery or double-open bug ships silently.

**Design.** Add a fake-worker-backed test for the three behaviours that matter: (1) session reuse across concurrent `/browse` requests (one shared session, not one per request), (2) fault recovery — after the backing session faults, the next request re-opens rather than throwing, (3) disposal releases the session. `FakeWorkerHarness` already models faults, so no live COM needed.

**Implementation.** New `DashboardLiveDataServiceTests` under `src/ZB.MOM.WW.MxGateway.Tests/Gateway/Dashboard/`. Optionally add hub-method tests for `EventsHub`/`AlarmsHubPublisher` group join/leave. Docs: none required (internal test coverage).

**Verification.** `dotnet test ... --filter FullyQualifiedName~DashboardLiveDataServiceTests`.

---

## TST-07 — Real-clock sleeps with negative assertions are latent flakes   `Medium` · `—`

**Finding.** Verified `src/ZB.MOM.WW.MxGateway.Tests/Gateway/Workers/WorkerClientTests.cs:431` sleeps `Task.Delay(150ms)` ("give the heartbeat monitor a few real check-intervals") then asserts `client.State == Ready` (state did *not* change); `SessionManagerTests.cs:400` races a 50 ms delayed state flip against a bounded wait. (Report cited :433 and :402; the sleep is at :431 and the race setup at :400 — minor drift.) 47 `Thread.Sleep`/`Task.Delay` occurrences across 22 test files, most benign.

**Impact.** Fixed-real-time negative assertions can pass spuriously (the awaited thing hadn't happened yet anyway) or fail under CI load — corrosive in an already-manual regime, and doubly so once TST-03 runs them on shared runners.

**Design.** Convert the negative-assertion sleeps to `ManualTimeProvider`-driven check-interval pumping (advance the fake clock across the monitor's interval deterministically) or poll-until-stable with a short quiescence window. The heartbeat monitor already accepts an injected clock in production, so `WorkerClientTests` can advance it instead of sleeping. Leave the benign cancellation-bounded polls and `ManualTimeProvider` clock uses alone.

**Implementation.** `WorkerClientTests.cs` (heartbeat within-ceiling case) and `SessionManagerTests.cs` (state-flip race) — drive via the injected `TimeProvider`. Docs: none.

**Verification.** `dotnet test ... --filter FullyQualifiedName~WorkerClientTests` and `~SessionManagerTests`; run each 10× to confirm no spurious pass/fail.

---

## TST-08 — Full-suite orphaned testhost processes   `Medium` · `P1`

**Finding.** CLAUDE.md ("Source Update Workflow") documents that the full gateway suite "leaves orphaned testhost processes" and mandates filtered runs as a workaround. The workaround is procedural and is now baked into the repo's memory (`feedback_targeted_tests_only.md`).

**Impact.** Any future CI (TST-03) or an unaware contributor inherits zombie processes; the leak points at an undisposed `NamedPipeServerStream` or a hosted-service test fixture that never stops.

**Design.** Find and fix the leaking fixture rather than institutionalizing the workaround. Method: run the full suite locally, enumerate surviving `testhost`/pipe handles, bisect by test collection to the leaking fixture (prime suspects: fake-worker harness pipe servers, `SessionLeaseMonitorHostedService`/`OrphanWorkerCleanupHostedService` test hosts that start a timer but are never `StopAsync`/`Dispose`d). Ensure every `IAsyncDisposable`/`IDisposable` fixture is disposed and every started hosted service is stopped in teardown. This is prerequisite to a reliable CI full-suite run; keep the filtered-run guidance until the leak is closed.

**Implementation.** Likely `src/ZB.MOM.WW.MxGateway.Tests` fixtures under `Gateway/Workers/Fakes/` and any hosted-service test. Docs: once fixed, update CLAUDE.md "Source Update Workflow" to drop the "leaves orphaned testhost processes" rationale (keep filtered-run guidance for speed, not correctness).

**Verification.** Run the full `dotnet test src/ZB.MOM.WW.MxGateway.Tests/...` once and confirm no surviving `testhost` processes (`pgrep -f testhost` empty after exit). Note the pre-existing macOS `SessionWorkerClientFactoryFakeWorkerTests` timeout-message failures are environmental and out of scope (`project_macos_pipe_timeout_test_failures.md`).

---

## TST-09 — Health checks cover only the auth store   `Medium` · `—`

**Finding.** Verified `src/ZB.MOM.WW.MxGateway.Server/GatewayApplication.cs:71-75` registers exactly one `AuthStoreHealthCheck` tagged `Ready`. No readiness check for Galaxy SQL reachability, LDAP bind, alarm-monitor session health, or worker-executable presence/launchability — each a documented startup dependency.

**Impact.** `/health` reports Ready while the gateway cannot open a session (missing/invalid worker exe) or browse Galaxy (SQL down). Deploys look healthy and fail on first real request.

**Design.** Add tagged health checks next to the existing one, cheapest-first:
- **Worker-executable check** — reuse `WorkerExecutableValidator` (already unit-tested); pure filesystem/PE check, no side effects. Tag `Ready`.
- **Galaxy SQL** and **LDAP bind** — network checks; cache the result (e.g. 30 s) so `/health` polling doesn't hammer SQL Server / GLAuth. Tag `Ready` (or a separate `Dependencies` tag if operators want liveness vs readiness split).
- Alarm-monitor session health — expose the monitor's current state as a check if the alarm feature is enabled.

Gate the network checks behind config so a gateway with dashboard/alarms disabled doesn't fail readiness on an unconfigured dependency.

**Implementation.** `GatewayApplication.cs` health-check registration; new check classes under `src/ZB.MOM.WW.MxGateway.Server/` (e.g. `Health/WorkerExecutableHealthCheck.cs`, `Health/GalaxySqlHealthCheck.cs`, `Health/LdapHealthCheck.cs`). Tests: `...Tests/Gateway/Health/`. Docs: `docs/GatewayConfiguration.md` and `gateway.md` health section — list the checks and their tags; note the Wonder deployment exposes only `/health/live` (`project_wonder_deployment.md`) so keep a liveness-only endpoint working.

**Verification.** `dotnet build src/ZB.MOM.WW.MxGateway.Server`; unit tests for each check; manual `curl /health/ready` with a missing worker exe returns Unhealthy.

---

## TST-10 — Deployment/upgrade undocumented and hand-rolled   `Medium` · `—`

**Finding.** Deployments are NSSM-wrapped services with configuration in NSSM environment variables (acknowledged in `docs/GatewayConfiguration.md` and the project-memory deploy notes). `scripts/` has packaging and e2e scripts but no deploy/upgrade script, and no runbook for the `MxAccessGw`→`OtOpcUa` service-dependency dance.

**Impact.** Deploys are tribal knowledge (captured only in the operator's private memory files, `project_deploy_mechanics.md`); a second operator cannot reproduce a deploy or upgrade from the repo.

**Design.** Commit `docs/Deployment.md` (runbook) plus a publish/deploy script that captures the NSSM env-var configuration as code. The runbook records: framework-dependent publish command, exe layout under `C:\publish\mxaccessgw\{Server,Worker}`, the ports (5120 gRPC / 5130 dashboard), the `Stop-Service -Force` + manual `OtOpcUa` restart sequence, and the Wonder-host HTTPS/self-signed specifics. The script parameterizes host + env-var set. This is documentation/scripting only — no source change, and it must not embed secrets (LDAP service password, API keys) per CLAUDE.md.

**Implementation.** New `docs/Deployment.md`; new `scripts/deploy-gateway.ps1` (idempotent: stop, copy, apply NSSM env vars from a config file, restart, restart dependents). Reference — do not copy secrets from — the operator memory notes. Docs cross-link from `gateway.md`.

**Verification.** Dry-run the script against windev (`10.100.0.48`); confirm the service restarts and `OtOpcUa` comes back. No unit tests (ops script).

---

## TST-11 — No version discipline on server; client versions drift   `Medium` · `P2`

**Finding.** Verified: Server/Worker csproj carry no `Version` (assemblies stamp 1.0.0); Contracts is `0.1.2` (`ZB.MOM.WW.MxGateway.Contracts.csproj:10`); Python `0.1.2` (`pyproject.toml:9`), Rust `0.1.2` (`Cargo.toml:3`), Java `0.2.0` (`build.gradle:16`) — and the Java bump required a hand-maintained `CLIENT_VERSION` constant that drifted once (commit `b6a0d90`).

**Impact.** Support cannot correlate a deployed gateway or a client artifact to a commit; duplicated version constants have already drifted.

**Design.** Single-source versions:
- **.NET side:** define `<Version>`/`<InformationalVersion>` in `src/Directory.Build.props` (which already exists and enforces build policy), stamping the git SHA into the informational version. This covers Server, Worker, Contracts uniformly.
- **Clients:** generate the client version constant at build from the package manifest rather than hand-maintaining it (Java's `CLIENT_VERSION` is the proven failure). Where a build-time generation step is impractical, add the descriptor-freshness-style check (TST-03) that fails if the constant ≠ the manifest version.
- Decide one version cadence: either lockstep all five clients + contracts, or document why Java leads (JDK-17 retarget shipped as 0.2.0). Recommendation: bring Python/Rust/dotnet/contracts to 0.2.0 to converge, then move in lockstep.

**Implementation.** `src/Directory.Build.props`; `clients/java/build.gradle` (generate constant from `version`); analogous generation/check for other clients. Docs: `docs/ClientPackaging.md` and each client README note the single-sourcing. Coordinate with the `CLI` domain report on client-version surface.

**Verification.** `dotnet build src/ZB.MOM.WW.MxGateway.slnx` and confirm assembly `InformationalVersion` carries the SHA (`dotnet --info` / reflection); per-client build confirms the constant matches the manifest.

---

## TST-12 — CLAUDE.md misstates default retention behaviour   `Medium` · `P0`

**Finding.** Verified `CLAUDE.md:79` states "Default config preserves the original single-subscriber, no-retention behavior," but code defaults enable retention: `DetachGraceSeconds = 30` (`Configuration/SessionOptions.cs:46`), `ReplayBufferCapacity = 1024` (`Configuration/EventOptions.cs:21`), `ReplayRetentionSeconds = 300` (`EventOptions.cs:29`). Only `AllowMultipleEventSubscribers` defaults off.

**Impact.** An agent or operator following CLAUDE.md assumes detached sessions die immediately and nothing is buffered — the opposite of reality, and it obscures the TST-02 security window that is on by default.

**Design.** Straight documentation correction (the roadmap lists this as P0 alongside owner re-validation). Replace the sentence with an accurate one.

**Implementation.** `CLAUDE.md:79` — change:

> Default config preserves the original single-subscriber, no-retention behavior.

to:

> Default config is single-subscriber (`AllowMultipleEventSubscribers` off), but detach-grace and replay retention are **on** by default (`DetachGraceSeconds=30`, `ReplayBufferCapacity=1024`, `ReplayRetentionSeconds=300`): a detached session is retained for 30 s and recent events are buffered for reconnect.

If TST-02's interim mitigation (flip retention off) is chosen instead of implementing owner re-validation, this sentence must instead document the flipped defaults — keep the two changes in the same commit. Docs-change-in-same-commit rule applies.

**Verification.** Read-back; no build. Cross-check the reworded claim against `SessionOptions.cs`/`EventOptions.cs` defaults.

---

## TST-13 — gateway.md carries stale design-era sketches   `Medium` · `P2`

**Finding.** Verified in `gateway.md`:
- The `WorkerEnvelope` snippet (~`:291-309`) shows `uint64 correlation_id = 4` and body tags `worker_hello=10 … command=20 … fault=26` with **no** `WorkerShutdownAck`. The actual proto uses `string correlation_id = 4` and `gateway_hello = 10 … worker_command = 13 … worker_shutdown_ack = 17 … worker_fault = 20` (`src/ZB.MOM.WW.MxGateway.Contracts/Protos/mxaccess_worker.proto:24,27,30,34,37`).
- `gateway.md:738` still states the gateway "allow[s] one active client event subscriber per session, reject[s] a second subscriber" as unconditional policy — contradicted by config-gated fan-out later in the same file and by `docs/Sessions.md`.
- `gateway.md:330` includes `rpc Session(stream ClientMessage) returns (stream ServerMessage)` which was never built; the real service is unary + server-streaming only (`mxaccess_gateway.proto:18-22`, no bidi RPC).

**Impact.** gateway.md is the mandated pre-change reading; internally inconsistent load-bearing sections cost every reader a reconciliation pass and can mislead a change.

**Design.** Don't re-inline the wire types (they drift). Replace the `WorkerEnvelope` sketch with a one-line pointer to `mxaccess_worker.proto` as the source of truth. Mark the single-subscriber paragraph as the *default* mode with a pointer to the fan-out section and `docs/Sessions.md`. Label the `Session` bidi RPC explicitly as unbuilt/deferred future work (see TST-23).

**Implementation.** `gateway.md` — three edits at the cited locations. Docs-only; no source change. This is part of the roadmap's P2 documentation-drift sweep and should land with the wider sweep (cookie name, GroupToRole sample, Java 17, Grpc.md RPC table) tracked in the `SEC`/`IPC`/`CLI` domains — cross-reference so the sweep is one commit where practical.

**Verification.** Read-back against the three proto/code citations above.

---

## TST-14 — Repo-root working artifacts need triage   `Medium` · `P2`

**Finding.** Verified: `MxAccessGateway-docs-{issues,fixed,final}.md` and `MxGatewayClient-docs-{issues,fixed}.md` are **untracked** (`git status` shows them untracked; `*-docs-{issues,fixed,final}.md` are gitignored at `.gitignore:152-154`), largest 586 KB / ~15 k lines. `stillpending.md` (tracked) is a 2026-06-15 audit snapshot serving as the de-facto backlog; `oldtasks.md` (tracked) is a human-readable mirror of the epic tasks.json; `REVIEW-PROCESS.md` + `code-reviews/` is a completed review system; `A2-galaxyrepository-adoption-handoff.md` is a completed migration handoff referenced from CLAUDE.md.

**Impact.** The dead untracked `*-docs-*.md` files (15 k+ lines) trip greps and agents. The tracked snapshots blur "backlog" vs "historical record."

**Design.** Triage by evidence, not blanket deletion:
- **Delete** the five untracked `*-docs-*.md` working files — dead local outputs of a finished docs-review pass. They are gitignored, so deletion is local only.
- **`stillpending.md`:** either promote to a maintained register (drop the "Generated: … Commit:" snapshot framing) or migrate its open items (§1.3, §1.4, §3.x, epic remainder) to the Gitea tracker `docs/ImplementationPlanIndex.md` describes. Recommendation: migrate, since a tracker already exists in the plan.
- **`oldtasks.md`:** keep only until the epic resumes/closes (TST-04), then delete — it is a mirror of the tasks.json.
- **`REVIEW-PROCESS.md` + `code-reviews/`:** keep (living, self-consistent, zero open findings).
- **`A2-galaxyrepository-adoption-handoff.md`:** archive under `docs/plans/` once its cross-repo follow-ups land; update the CLAUDE.md reference if moved.

**Implementation.** `rm` the untracked files; move/rewrite `stillpending.md`; update CLAUDE.md if `A2-…` moves. No source change.

**Verification.** `git status` clean of the `*-docs-*.md` files; `grep -rn` for those filenames returns nothing.

---

## TST-15 — Dashboard EventsHub has no per-session ACL   `Medium` · `P2`

**Finding.** Verified `src/ZB.MOM.WW.MxGateway.Server/Dashboard/Hubs/EventsHub.cs:39` carries `TODO(per-session-acl)` — the only production-source TODO in the repo. Any authenticated dashboard Viewer may join any `session:{id}` group (group name built at `EventsHub.cs:21`) and observe that session's event stream.

**Impact.** Acceptable for a single-tenant dashboard; wrong the moment `GroupToRole` admits low-trust viewers. It is the dashboard-side twin of the gRPC owner-revalidation gap (TST-02).

**Design.** Covered by epic Phase 4 (Tasks 16–19, TST-04). Introduce a role/scope that scopes a hub connection to permitted sessions, plus a hub-token session tag (Task 18). The open design decision (`oldtasks.md:52`) — Viewer default of admin-sees-all vs strict per-session — must be settled first. Recommendation: admin-sees-all, Viewer strictly scoped to sessions they own/are granted, matching TST-02's gRPC owner binding for consistency. Until Phase 4 lands, keep the TODO (it correctly documents the accepted single-tenant assumption); do not silently remove it.

**Implementation.** `Dashboard/Hubs/EventsHub.cs` (ACL check on group join), hub-token minting to carry the session tag, `Configuration/DashboardOptions.cs` for any group-to-tag config (Task 17). Tests: `...Tests/Gateway/Dashboard/` hub ACL cases incl. live-LDAP users (Task 19). Docs: `docs/Sessions.md`/`gateway.md` dashboard section document the ACL model; CLAUDE.md dashboard-auth paragraph.

**Verification.** `dotnet test ... --filter FullyQualifiedName~EventsHub`; `dotnet build src/ZB.MOM.WW.MxGateway.Server`.

---

## TST-16 — `Dashboard:ShowTagValues` is a dead flag   `Medium` · `—`

**Finding.** Verified bound at `Configuration/DashboardOptions.cs:62`, projected into effective config (`EffectiveDashboardConfiguration.cs:9`, `GatewayConfigurationProvider.cs:63`), and displayed read-only on the settings page (`Dashboard/Components/Pages/SettingsPage.razor:65`) — but it gates no value-display behaviour anywhere. `docs/GatewayConfiguration.md:174` calls it "Reserved". (Note `docs/GatewayDashboardDesign.md:305` claims it "continues to govern" value display — that prose is aspirational/stale.)

**Impact.** An operator toggling a flag that renders on the settings page reasonably expects an effect; there is none.

**Design.** Decide implement-or-remove. Recommendation: **implement** — the flag has an obvious intended meaning (gate live tag-value display in `/browse`, consistent with the redaction philosophy in CLAUDE.md "Don't log secrets or full tag values by default"). Wire `ShowTagValues` into `DashboardLiveDataService`/the browse value-rendering path so `false` suppresses live values. If the team prefers not to implement, remove it from the options, the projector, the settings page, and downgrade the docs row — a settings-page-visible no-op is the worse outcome. Open question: default value (currently `false`) and whether it interacts with the per-session ACL (TST-15).

**Implementation.** `Dashboard/DashboardLiveDataService.cs` + the browse value component to honour the flag (implement path); or remove from `DashboardOptions.cs:62`, `EffectiveDashboardConfiguration.cs`, `GatewayConfigurationProvider.cs`, `SettingsPage.razor` (remove path). Tests: extend the TST-06 `DashboardLiveDataServiceTests` for the flag. Docs: `docs/GatewayConfiguration.md:174` and `docs/GatewayDashboardDesign.md:305` — replace "Reserved"/stale prose with the real behaviour.

**Verification.** `dotnet build src/ZB.MOM.WW.MxGateway.Server`; test toggling the flag suppresses/shows values; docs match.

---

## TST-17 — Vendor-gated alarm parity residuals silently lossy   `Medium` · `—`

**Finding.** Verified `src/ZB.MOM.WW.MxGateway.Worker/MxAccess/WnWrapAlarmConsumer.cs:261-278`: the 8-arg `AlarmAckByName` `domain`/`full_name` fields are accepted on the wire then wrapped into the 6-arg call because the 8-arg form returns −55 on this AVEVA build (comments at `:119`, `:265-270`). `AlarmAckByGUID` is `E_NOTIMPL`. The `provider_switches` metric reason tagging has never been exercised live (`stillpending.md` §1.3–1.4, §3.4–3.5).

**Impact.** Contract fields that silently do nothing, invisible to callers — a parity gap dressed as a working feature.

**Design.** This is a genuine vendor limitation, not a bug to "fix" — per CLAUDE.md, MXAccess behaviour is the baseline and must not be papered over. The remediation is **observability, not behaviour change**: surface the drop so callers know. Add a diagnostic string to the ack reply (e.g. reply `message`/diagnostic field) noting that operator `domain`/`full_name` were not applied on this build (8-arg returns −55), and that `AlarmAckByGUID` is `E_NOTIMPL`. Do not synthesize success or invent a result — just report the degrade. Keep the TODO/comment referencing the AVEVA stub so a future runtime that implements it can drop the workaround.

**Implementation.** `Worker/MxAccess/WnWrapAlarmConsumer.cs` — populate a diagnostic in the ack reply when the 8-arg path is downgraded. Confirm the gateway forwards that diagnostic to the client reply (`Gateway/Grpc/...`). Tests: worker ack-consumer test asserting the diagnostic is present on the downgraded path (`src/ZB.MOM.WW.MxGateway.Worker.Tests`). Docs: `gateway.md`/`docs/` alarm section and the relevant client READMEs note the vendor-gated residual. Cross-reference the `WRK` domain report for the alarm-consumer specifics.

**Verification.** `dotnet build src/ZB.MOM.WW.MxGateway.Worker -p:Platform=x86`; `dotnet test src/ZB.MOM.WW.MxGateway.Worker.Tests/... -p:Platform=x86` (Windows).

---

## TST-18 — Hosted-service wrappers untested   `Low` · `—`

**Finding.** `SessionLeaseMonitorHostedService` and `OrphanWorkerCleanupHostedService` delegate to well-tested cores (sweep logic covered via `SessionManagerTests.CloseExpiredLeasesAsync` cases), but their timer/startup wiring has no direct tests.

**Impact.** Low — failure mode (a timer that never fires) is obvious at startup and in integration.

**Design.** Add a thin `StartAsync` → tick-once (via injected `TimeProvider`) → assert the core method was invoked → `StopAsync` disposes cleanly test for each. This also directly exercises the disposal path relevant to TST-08 (verify the hosted service stops its timer on `StopAsync`), so it doubles as a leak-regression guard.

**Implementation.** New small test classes under `src/ZB.MOM.WW.MxGateway.Tests/Gateway/Sessions/` and `.../Workers/`. Docs: none.

**Verification.** `dotnet test ... --filter FullyQualifiedName~HostedService`.

---

## TST-19 — Keep FakeWorkerHarness canned replies in lockstep   `Low` · `—`

**Finding.** `FakeWorkerHarness` shares `WorkerFrameReader`/`WorkerFrameWriter`/`WorkerEnvelope` with production and scripts malformed/oversized frames, heartbeats, faults, shutdown acks, and control-command reply shapes. What it cannot represent (STA pumping, COM HRESULT semantics, process exit codes, event timing under load) is honestly delegated to the live suite. History (`stillpending.md` §1.1) shows the canned replies drifted once: green fakes masked an unimplemented real worker for months.

**Impact.** Low today; the risk is future drift where fake replies stop matching `WorkerPipeSession`, hiding a real-worker regression.

**Design.** No code fix — this is a process guard. The durable protection is TST-05's scheduled live suite (it catches fake/real drift automatically) plus a code-review checklist note. Add a comment/README note in the fakes directory pointing at `WorkerPipeSession` as the reply-shape source of truth, so any control-command change updates both.

**Implementation.** `src/ZB.MOM.WW.MxGateway.Tests/Gateway/Workers/Fakes/` — add a short README or header comment. Rely on TST-05 (scheduled live) for the real guard. Docs: `docs/GatewayTesting.md` already states the fidelity charter; add the "keep in lockstep, live suite catches drift" note.

**Verification.** Covered by TST-05's scheduled live run; no standalone test.

---

## TST-20 — E2E script papers over a real advise-without-consumer sharp edge   `Low` · `—`

**Finding.** `scripts/run-client-e2e-tests.ps1` (~1,715 lines) interleaves `StreamEvents` drains every ~15 advised tags because advising without a consumer overflows the worker event channel and faults the session under `FailFast` (`docs/GatewayTesting.md` "Client E2E Scripts" phase 3).

**Impact.** Low as a test detail, but it is real product feedback: any client doing bulk-advise-then-stream will fault sessions.

**Design.** Treat as product feedback, not a script fix. The clean solution is a subscribe-time channel policy (buffer/backpressure the event channel until a subscriber attaches, or a bounded pre-subscription buffer) — but that is a cross-domain backpressure decision owned by the `IPC`/`GWC`/`WRK` reports' "coordinated size/backpressure pass" (roadmap P1 #8). This finding's action is to **record** the sharp edge as an input to that pass and, until then, document the advise-then-stream ordering requirement in the client READMEs so real clients don't fault sessions.

**Implementation.** No script change. Docs: client READMEs and `gateway.md` event section note "attach `StreamEvents` before bulk advise, or drain periodically." Cross-reference the backpressure-pass finding in the other domains.

**Verification.** Docs read-back; the behavioural fix is verified under the coordinated backpressure pass, not here.

---

## TST-21 — Log rotation configured but minimal   `Low` · `—`

**Finding.** Verified `src/ZB.MOM.WW.MxGateway.Server/appsettings.json:8-11` rolls the Serilog file sink daily (`logs/mxgateway-.log`, relative to the service working directory) with no `retainedFileCountLimit` or size cap (Serilog defaults: 31 files, unbounded size per day).

**Impact.** Acceptable, but a high-rate event day can produce an unbounded single file, and a relative path lands logs in the service CWD.

**Design.** Set `fileSizeLimitBytes` + `rollOnFileSizeLimit: true` and an absolute log path for the deployed service. Keep it in the deployment config (NSSM env-var overlay / deploy script from TST-10) rather than only in the committed `appsettings.json`, since the absolute path is host-specific.

**Implementation.** `src/ZB.MOM.WW.MxGateway.Server/appsettings.json` Serilog file sink (add size cap + retained-count) and the TST-10 deploy config for the absolute path. Docs: `docs/GatewayConfiguration.md` logging notes; `docs/Deployment.md`.

**Verification.** `dotnet run --project src/ZB.MOM.WW.MxGateway.Server/...` and confirm rollover on size; no unit test.

---

## TST-22 — Config-shape JSON block omits documented keys   `Low` · `—`

**Finding.** Verified the Configuration Shape JSON block (`docs/GatewayConfiguration.md:12-83`) omits `DisableLogin`/`AutoLoginUser`/`CookieName`, the `Tls` block, and the `Alarms:Fallback` block — all of which the tables below it document. (The rest of `docs/GatewayConfiguration.md` is current: table defaults match `Configuration/*.cs`.)

**Impact.** Low — the tables are authoritative and correct; the shape block is a convenience that is now incomplete, so a copy-paste starting config misses those keys.

**Design.** Add the missing keys to the JSON block so it round-trips with the tables. Pure documentation.

**Implementation.** `docs/GatewayConfiguration.md:12-83` — add `Dashboard:DisableLogin`, `Dashboard:AutoLoginUser`, `Dashboard:CookieName`, the `Tls` section, and `Alarms:Fallback`. Verify against `Configuration/DashboardOptions.cs`, `TlsOptions.cs`, and the alarms fallback options.

**Verification.** Read-back against the cited options classes.

---

## TST-23 — Bidirectional `Session` RPC never built   `Low` · `P2`

**Finding.** Verified `gateway.md:330` presents `rpc Session(stream ClientMessage) returns (stream ServerMessage)` as the "best long-term shape"; the shipped service stopped at unary + server-streaming (`mxaccess_gateway.proto:18-22`, no bidi RPC).

**Impact.** None functional — purely a docs-currency/roadmap-clarity issue (the twin of TST-13).

**Design.** Fold into the TST-13 gateway.md edit: label the `Session` RPC explicitly as unbuilt future work (or remove it if it is no longer the intended direction). Decide whether bidi `Session` is still on the roadmap; if not, drop the "best long-term shape" framing so it doesn't read as an imminent deliverable.

**Implementation.** `gateway.md` around `:328-345` — one clarifying edit, made with TST-13. Docs-only.

**Verification.** Read-back; consistent with `mxaccess_gateway.proto`.

---

## TST-24 — Client wire behaviour has no automated verification   `Low` · `—`

**Finding.** All five clients have unit tests (13/8/3/13/7 files for dotnet/go/rust/python/java) but no in-process or containerized gateway integration tests; the only cross-language verification is the operator-run `scripts/run-client-e2e-tests.ps1`. `CrossLanguageSmokeMatrixTests` checks shapes only.

**Impact.** Low-to-moderate: a gateway contract change can pass every default suite and break all five clients (partly mitigated by shared-proto codegen). The Java CLI already proves the cheap pattern — `InProcessGatewayHarness` (`stillpending.md` §8).

**Design.** Replicate the in-process fake-gateway pattern per client so each client's happy path (open → invoke → stream) runs against a lightweight fake gateway in the client's own test suite, wired into CI (TST-03). Java has the template; port the idea to the other four. This is the cheapest way to catch contract breaks without a live rig. Full parity is out of scope — target the core session/invoke/stream shapes and the `ReplayGap` handling added in TST-01.

**Implementation.** Per-client test harness (e.g. an in-process gRPC server serving canned replies) under each `clients/<lang>/` test tree; add to the CI client jobs. Coordinate the fake-gateway contract with the `CLI` domain report. Docs: each client README notes the harness.

**Verification.** Per-client test command (CLAUDE.md client table); the harness tests run inside CI (TST-03).