docs(archreview): 2026-07-12 follow-up review + remediation designs

Six-domain re-review at the P2 merge (4f5371f): all prior Done claims
verified (none false, 10 partial), 47 new findings (1 High, 14 Medium),
with per-finding design/implementation plans and a new tracking register.

archreview/2026-07-12/remediation/00-tracking.md

@@ -0,0 +1,158 @@
+# MxAccessGateway — Remediation Tracking (2026-07-12 review)
+
+Master progress tracker for the 2026-07-12 follow-up architecture review. Generated 2026-07-13.
+
+Source review: [`../00-overall.md`](../00-overall.md) · Per-domain remediation designs are linked below and hold the full **Finding / Impact / Design / Implementation / Verification** for every entry here. The first-cycle tracker ([`../../remediation/00-tracking.md`](../../remediation/00-tracking.md)) remains the record for the original 153 findings; this document tracks only the 47 new IDs (GWC-24+, WRK-21+, IPC-23+, SEC-31+, CLI-35+, TST-25+) plus the old-tracker actions listed at the end.
+
+## How to use this document
+
+- Each finding has a stable ID that never changes. Cite it in commits, branches, and PRs (e.g. `fix(GWC-25): empty-ring ReplayGap sentinel`).
+- The **Status** column is the single source of truth for progress. Update it in the same change that lands the fix.
+- Do the work in **roadmap-tier order** (P0 → P1 → P2), respecting the `Dep` column and the cross-cutting clusters below — several fixes are one change set across two domains and must land together.
+- When a fix lands: flip Status to `Done` and, per the repo rule, update the affected docs in the same commit.
+
+**Status legend:** `Not started` · `In progress` · `In review` · `Done` · `Won't fix` (record why in the domain doc) · `N/A` (informational / decision recorded, no action).
+
+## Severity roll-up
+
+| Domain | Doc | High | Medium | Low | Info | Total |
+|---|---|:-:|:-:|:-:|:-:|:-:|
+| Gateway core | [10-gateway-core.md](10-gateway-core.md) | — | 2 | 4 | 1 | 7 |
+| Worker | [20-worker.md](20-worker.md) | — | 1 | 7 | — | 8 |
+| Contracts & IPC | [30-contracts-ipc.md](30-contracts-ipc.md) | — | 3 | 5 | 2 | 10 |
+| Security & dashboard | [40-security-dashboard.md](40-security-dashboard.md) | — | 1 | 4 | 1 | 6 |
+| Clients | [50-clients.md](50-clients.md) | — | 5 | 6 | — | 11 |
+| Testing, docs & gaps | [60-testing-docs-gaps.md](60-testing-docs-gaps.md) | 1 | 2 | 2 | — | 5 |
+| **Total** | | **1** | **14** | **28** | **4** | **47** |
+
+## Roadmap-tier roll-up
+
+| Tier | Meaning | Count | Findings |
+|---|---|:-:|---|
+| **P0** | Correctness & safety — all small-to-medium | 10 | GWC-25, WRK-21, IPC-23, IPC-24, IPC-25, IPC-30, SEC-31, SEC-32, CLI-35, CLI-36 |
+| **P1** | Process & hardening | 12 | GWC-24, WRK-26, SEC-33, SEC-36, CLI-37, CLI-38, CLI-39, CLI-42, CLI-45, TST-25, TST-26, TST-27 |
+| **P2** | Completeness & polish | 9 | GWC-26, GWC-27, GWC-28, WRK-25, IPC-26, IPC-27, SEC-34, TST-28, TST-29 |
+| **—** | Not individually in the roadmap (Lows/Infos) | 16 | GWC-29/30, WRK-22/23/24/27/28, IPC-28/29/31/32, SEC-35, CLI-40/41/43/44 |
+
+### P0 — do first (10)
+
+Sequenced by cluster; a cluster is one change set.
+
+| ID | Sev | Eff | Dep | Status | Title |
+|---|---|:-:|---|---|---|
+| GWC-25 | Medium | S | CLI-35/36 (coord) | Not started | Empty-ring ReplayGap sentinel carries `oldest_available_sequence = 0`, dead-streaming a compliant client |
+| CLI-35 | Medium | S | GWC-25 (coord) | Not started | Python CLI `stream-events` crashes on a ReplayGap |
+| CLI-36 | Medium | S | GWC-25 (coord) | Not started | Go CLI `stream-events` silently destroys the ReplayGap signal |
+| WRK-21 | Medium | M | owns IPC-23 fix; WRK-28 same batch | Not started | DrainEvents bound is count-based only; oversized reply kills the session and loses the drained events |
+| IPC-23 | Medium | S | WRK-21 | Not started | DrainEvents contract requirements (reply fits negotiated max, no event loss, drain-until-empty) + proto-comment/doc wave |
+| IPC-30 | Low | M | WRK-21 (same batch) | Not started | Oversized event frame stays session-fatal by design, but the death becomes structured (fault frame + logged identity) |
+| SEC-31 | Medium | M | — | Not started | Failure limiter partitions on attacker-controlled key id and blocks before verification (lockout DoS) |
+| SEC-32 | Low | S | SEC-31 | Not started | Failure-limiter LRU flushable by junk-token spray; token prefix never validated |
+| IPC-24 | Medium | S | — | Not started | CI's unconditional Java churn-revert masks real drift |
+| IPC-25 | Medium | M | — | Not started | Regenerate stale Go/Python worker bindings + add binding-freshness guard (Check 4) to check-codegen.ps1 |
+
+## Finding registers by domain
+
+Full design + implementation for each row lives in the linked domain doc under its ID heading.
+
+### Gateway core — [10-gateway-core.md](10-gateway-core.md)
+
+| ID | Sev | Tier | Eff | Dep | Status | Title |
+|---|---|:-:|:-:|---|---|---|
+| GWC-24 | Medium | P1 | M | GWC-21 (coord, old tracker) | Not started | Unbounded event staging channel: sustained slow drain grows memory silently and invisibly |
+| GWC-25 | Medium | P0 | S | CLI-35/36 (coord) | Not started | Empty-ring ReplayGap sentinel carries `oldest_available_sequence = 0` |
+| GWC-26 | Low | P2 | M | GWC-27 | Not started | Alarm monitor attaches its subscriber after SubscribeAlarms; window transitions bypass the feed |
+| GWC-27 | Low | P2 | S | GWC-26 | Not started | `AttachInternalEventSubscriber` bypasses the readiness gate; premature attach poisons the distributor |
+| GWC-28 | Low | P2 | S | GWC-10 (coord, old tracker) | Not started | Gateway→worker envelope `sequence` stamped at creation, not at write |
+| GWC-29 | Low | — | S | — | Not started | `Invoke` deep-clones the entire request only to discard the cloned command |
+| GWC-30 | Info | — | S | — | Not started | Frame reader allocates a fresh 4-byte length-prefix array per frame |
+
+### Worker — [20-worker.md](20-worker.md)
+
+| ID | Sev | Tier | Eff | Dep | Status | Title |
+|---|---|:-:|:-:|---|---|---|
+| WRK-21 | Medium | P0 | M | IPC-23 (fix owned here); WRK-28 | Not started | DrainEvents bound count-based only; oversized reply kills session and loses drained events |
+| WRK-22 | Low | — | S | IPC-26 (fix owned here) | Not started | Cancelled `WriteAsync` leaves its frame queued; it is still written later |
+| WRK-23 | Low | — | S | WRK-21 | Not started | Rejected frames consume sequence numbers, producing wire gaps |
+| WRK-24 | Low | — | S | — | Not started | `AdoptNegotiatedMaxMessageBytes` has no lower-bound sanity check |
+| WRK-25 | Low | P2 | S | WRK-22 (shared seam) | Not started | WRK-12 flush coalescing never engages on the event hot path |
+| WRK-26 | Low | P1 | S | WRK-23 (soft); discharges IPC-29 | Not started | Write-priority and overflow doc drift from the WRK-07 change |
+| WRK-27 | Low | — | S | — | Not started | Alarm poll bypasses the watchdog's in-flight suppression (15 s vs 75 s) |
+| WRK-28 | Low | — | S | WRK-21 (same batch) | Not started | 10,000 drain cap is a duplicated magic constant |
+
+### Contracts & IPC — [30-contracts-ipc.md](30-contracts-ipc.md)
+
+| ID | Sev | Tier | Eff | Dep | Status | Title |
+|---|---|:-:|:-:|---|---|---|
+| IPC-23 | Medium | P0 | S | WRK-21 (mechanics) | Not started | DrainEvents byte-blindness — contract requirements + proto-comment/doc wave |
+| IPC-24 | Medium | P0 | S | — | Not started | CI's unconditional Java churn-revert masks real drift |
+| IPC-25 | Medium | P0 | M | — | Not started | Stale Go/Python worker bindings: regenerate (pinned toolchains) + check-codegen Check 4 |
+| IPC-26 | Low | P2 | S | WRK-22 (mechanics) | Not started | Cancelled write leaves ghost frame — cancelled means never written |
+| IPC-27 | Low | P2 | S | — | Not started | Descriptor-freshness test blind to enums/services/galaxy descriptor |
+| IPC-28 | Low | — | S | — | Not started | docs/Grpc.md missing CommandTooLarge → ResourceExhausted mapping |
+| IPC-29 | Low | — | S | WRK-26 (discharged by) | Not started | WorkerFrameProtocol.md missing write-scheduling/sequencing section |
+| IPC-30 | Low | P0 | M | WRK-21 (same batch) | Not started | Oversized event frame: keep session-fatal, make the death structured |
+| IPC-31 | Info | — | — | — | N/A | Gateway creation-time sequence stamping accepted; diagnostic-only, decision recorded |
+| IPC-32 | Info | — | S | IPC-25 (folded in) | Not started | check-codegen banner relabel 1/4…4/4 |
+
+### Security & dashboard — [40-security-dashboard.md](40-security-dashboard.md)
+
+| ID | Sev | Tier | Eff | Dep | Status | Title |
+|---|---|:-:|:-:|---|---|---|
+| SEC-31 | Medium | P0 | M | — | Not started | Failure limiter: composite (peer, key-id) partitions + cross-peer aggregate with probe admission |
+| SEC-32 | Low | P0 | S | SEC-31 | Not started | Limiter LRU flushable by junk-token spray; validate token shape, cap per-peer partitions |
+| SEC-33 | Low | P1 | M | old SEC-23 (co-locate) | Not started | Host-meaningful path rooting; drop Windows literals from appsettings; validate Galaxy `SnapshotCachePath` |
+| SEC-34 | Low | P2 | S | — | Not started | Verification cache: expiry outlives TTL; `Invalidate` races in-flight repopulation |
+| SEC-35 | Info | — | S | — | N/A | Production hard-stops key on exact `Production` environment name (doc-only) |
+| SEC-36 | Low | P1 | M | cross-repo `scadaproj/infra/glauth` | Not started | Committed dev LDAP service-account password: rotate, remove, move dev channel to user-secrets |
+
+### Clients — [50-clients.md](50-clients.md)
+
+| ID | Sev | Tier | Eff | Dep | Status | Title |
+|---|---|:-:|:-:|---|---|---|
+| CLI-35 | Medium | P0 | S | GWC-25 (coord) | Not started | Python CLI `stream-events` crashes on a ReplayGap |
+| CLI-36 | Medium | P0 | S | GWC-25 (coord) | Not started | Go CLI `stream-events` silently destroys the ReplayGap signal |
+| CLI-37 | Medium | P1 | M | CLI-38 (co-land) | Not started | Status-array validation must branch on `category` per the proto contract (4-vs-1 divergence) |
+| CLI-38 | Medium | P1 | S | — | Not started | Align .NET/Go/Java on `hresult < 0` — lands old CLI-08, cures design-doc drift |
+| CLI-39 | Medium | P1 | S | CLI-35..38, CLI-45 (land last) | Not started | Bump client versions off published 0.1.2 (converge on 0.2.0); registry-collision guard in pack-clients.ps1 |
+| CLI-40 | Low | — | M | — | Not started | Port the exact-secret credential scrub to Rust/Java/.NET |
+| CLI-41 | Low | — | M | — | Not started | Uniform malformed-reply contract for AuthenticateUser/ArchestrAUserToId/AddBufferedItem |
+| CLI-42 | Low | P1 | S | — | Not started | Document the vendored Rust proto layout (CLI-02's missing doc half) |
+| CLI-43 | Low | — | S | — | Not started | Java style guide still prescribes "Java 21 preferred" |
+| CLI-44 | Low | — | S | — | Not started | Go event goroutine can mislabel a genuine terminal error as `ErrSlowConsumer` |
+| CLI-45 | Low | P1 | M | — | Not started | Standardize CLI credential env-var names; fail fast on missing/empty passwords |
+
+### Testing, docs & gaps — [60-testing-docs-gaps.md](60-testing-docs-gaps.md)
+
+| ID | Sev | Tier | Eff | Dep | Status | Title |
+|---|---|:-:|:-:|---|---|---|
+| TST-25 | High | P1 | M | unlocks old TST-05, TST-24 | Not started | Windows/x86 test tier has zero automation — SSH-driven windev CI job |
+| TST-26 | Medium | P1 | S | TST-25 (same commit) | Not started | Docs/scripts describe removed CI jobs; Generated/-guard reattributed to check-codegen |
+| TST-27 | Medium | P1 | S | — | Not started | `ShowTagValues` config row still says "Reserved" after SEC-25 made the flag live |
+| TST-28 | Low | P2 | S | relates IPC-02 (old) | Not started | Gateway-side `max_frame_bytes` handshake untested in the CI-run suite |
+| TST-29 | Low | P2 | S | — | Not started | Retire `oldtasks.md` (fold Phase-5 governance into DesignDecisions.md); delete root artifacts |
+
+## Cross-cutting clusters
+
+Sequence these together rather than piecemeal — several are one change set spanning two domains:
+
+- **Drain / oversized-frame cluster (P0):** WRK-21 + IPC-23 + IPC-30, with WRK-28 and WRK-23 in the same batch. One worker change set in `WorkerPipeSession.cs`/`WorkerFrameWriter.cs`; WRK-21 owns the byte-budgeted drain and must satisfy IPC-23's requirements R1–R3; IPC-30's structured-fault seam lands in the same drain loop. IPC-23's proto-comment edits trigger the full regen fan-out (Generated/, Rust vendored, Go/Java, descriptor set) — land with WRK-21 so the wave happens once. One windev x86 verification run for the cluster.
+- **ReplayGap end-to-end (P0):** GWC-25 (server sentinel arithmetic) + CLI-35 (Python CLI) + CLI-36 (Go CLI). Independently landable; the e2e resume walk validates only when all three are in.
+- **Auth limiter (P0):** SEC-31 + SEC-32 — same component, one change set, one test suite.
+- **Codegen freshness (P0):** IPC-24 + IPC-25 (+ IPC-32 folded in). Both edit `.gitea/workflows/ci.yml` — coordinate the branch with TST-25, which touches the same file.
+- **Windows-tier automation (P1):** TST-25 + TST-26 (same commit). Unlocks old TST-05/TST-24 and provides CI evidence for every windev-verified cluster above; until it lands, record windev runs in this tracker's change log.
+- **Client conformance + release train (P1):** CLI-37 + CLI-38 co-land (one conformance commit; closes old CLI-08), then CLI-45, with CLI-40/41 fixtures as follow-ups; **CLI-39 lands last** so published 0.2.0 carries the conformant behavior. Shared fixtures under `clients/proto/fixtures/behavior/`; update CrossLanguageSmokeMatrix.md/ClientLibrariesDesign.md same-commit. Do not republish regenerated bindings (IPC-25) before CLI-39 resolves.
+- **Doc-drift batch (P1):** TST-27 + WRK-26 (discharges IPC-29) + CLI-42 + IPC-28 + SEC-35's doc note — one sweep commit is fine.
+- **Backpressure follow-on (P1):** GWC-24, coordinating with still-open old GWC-21 (`EventChannelFullModeTimeout` configurability).
+
+## Old-tracker actions ([`../../remediation/00-tracking.md`](../../remediation/00-tracking.md))
+
+- Close **CLI-24** and **CLI-34** as `Done` (incidentally fixed; evidence in [../50-clients.md](../50-clients.md)).
+- When CLI-38 lands, close old **CLI-08** with a pointer here.
+- When WRK-26 lands, its doc section also discharges the WorkerFrameProtocol gap; when TST-25 lands, revisit old **TST-05** (scheduled live smoke) and **TST-24** (client wire tests), which it unlocks.
+
+## Change log
+
+| Date | Change |
+|---|---|
+| 2026-07-13 | Initial tracking doc generated from the six domain remediation designs. All 47 findings `Not started` (IPC-31, SEC-35 `N/A`). |

archreview/2026-07-12/remediation/10-gateway-core.md

@@ -0,0 +1,171 @@
+# Gateway Server Core — Remediation Design & Implementation (2026-07-12 cycle)
+
+Source review: [10-gateway-core.md](../10-gateway-core.md) · Roadmap: [00-overall.md](../00-overall.md) · Generated: 2026-07-13
+
+This document turns the 2026-07-12 re-review's **new** Gateway Server Core findings (GWC-24 through GWC-30) into buildable remediation entries. Every cited `path:line` was re-verified against `main` at `4f5371f`. Tiers follow the overall roadmap: GWC-25 is P0.1 (grouped with the client-side CLI-35/36 ReplayGap handling), GWC-24 is P1.6, and GWC-26/27/28 sit in the P2 batch; GWC-29/30 are untiered polish. Prior-cycle findings still open (GWC-05, 09, 10, 11, 12, 13, 16–22) are tracked in `archreview/remediation/10-gateway-core.md` and are not re-planned here, but two of them (GWC-10, GWC-21) are named below where a new fix should coordinate with them.
+
+## Finding index
+
+| ID | Sev | Tier | Eff | Dep | Status | Title |
+|----|-----|------|-----|-----|--------|-------|
+| GWC-24 | Medium | P1 | M | GWC-21 (coord) | Not started | Unbounded event staging channel: sustained slow drain grows memory silently and invisibly |
+| GWC-25 | Medium | P0 | S | CLI-35/36 (coord) | Not started | Empty-ring ReplayGap sentinel carries `oldest_available_sequence = 0`, dead-streaming a compliant client |
+| GWC-26 | Low | P2 | M | GWC-27 | Not started | Alarm monitor attaches its subscriber after SubscribeAlarms; window transitions bypass the feed, missed Acknowledge never repaired |
+| GWC-27 | Low | P2 | S | GWC-26 | Not started | `AttachInternalEventSubscriber` bypasses the readiness gate; premature attach poisons the distributor permanently |
+| GWC-28 | Low | P2 | S | GWC-10 (coord) | Not started | Gateway→worker envelope `sequence` stamped at creation, not at write — non-monotonic on the wire under concurrent invokes |
+| GWC-29 | Low | — | S | — | Not started | `Invoke` deep-clones the entire request only to discard the cloned command |
+| GWC-30 | Info | — | S | — | Not started | Frame reader allocates a fresh 4-byte length-prefix array per frame |
+
+Dependency notes: GWC-26 and GWC-27 both change the internal-subscriber attach path (`GatewaySession.AttachInternalEventSubscriber` and its `SessionManager`/alarm-monitor callers) — land GWC-27's readiness gate first (or in the same commit), then GWC-26's reorder, so the reordered monitor attach is proven against the gate. GWC-24 is the direct successor of the prior cycle's GWC-04 backpressure fix and raises the value of the still-open GWC-21 (making `EventChannelFullModeTimeout` configurable); GWC-28 is the gateway half of the worker's WRK-04 fix and must be coordinated with the still-open GWC-10 if inbound sequence enforcement is ever added. GWC-25 is server-complete on its own, but the end-to-end reconnect story also needs the client-domain CLI-35/36 fixes (Python CLI crashes on the sentinel, Go CLI destroys it).
+
+---
+
+## GWC-24 — Unbounded event staging channel: sustained slow drain grows memory silently and invisibly   `Medium` · `P1`
+
+**Finding.** The GWC-04 remediation decoupled the read loop from event backpressure by staging events into `_eventStaging`, an **unbounded** channel (`Workers/WorkerClient.cs:93-100`). `StageWorkerEvent`'s `TryWrite` therefore always succeeds (`:565-573`), and the sustained-overflow `ProtocolViolation` fault fires only when a *single* timed `WriteAsync` against the bounded `_events` exceeds `EventChannelFullModeTimeout` (default 5 s, `:610-647`). The queue-depth gauge counts only `_events` — `_eventQueueDepth` is incremented in `EnqueueWorkerEventAsync` (`:616`, `:627`) and decremented in `ReadEventsCoreAsync` (`:289`), so staged-but-unqueued events are invisible to `SetWorkerEventQueueDepth`. The field comment (`:28-33`) claims staging "only fills during the bounded EventChannelFullModeTimeout window", which is true only for a full consumer stall, not for a consumer that drains slower than the worker produces while each individual write still completes inside the window.
+
+**Impact.** Before GWC-04, a full `_events` blocked the read loop, which propagated backpressure through the pipe to the worker's own bounded queue — gateway memory per session was structurally bounded end-to-end. Now a slow-but-live distributor pump (each `WriteAsync` completing in, say, 4 s while the worker emits faster) grows `_eventStaging` without bound, without any fault, and without any metric showing it. The repo's fail-fast backpressure invariant currently rests on pump liveness (the pump's own fan-out is non-blocking `TryWrite`, `Sessions/SessionEventDistributor.cs:582-594`) rather than on channel bounds — unlikely to bite, but no longer structurally impossible, and undiagnosable when it does.
+
+**Design.** Do **both** halves of the review's recommendation — restore a structural bound *and* make the backlog observable:
+
+1. **Bound the staging channel** at `2 × EventChannelCapacity` (derived from the existing option; no new configuration key). Total gateway-side buffering per session becomes `3 × MxGateway:Events:QueueCapacity` (bounded consumer channel + 2× staging), which absorbs the same bursts the unbounded design absorbed in practice while capping the pathological case. Create it with `FullMode = BoundedChannelFullMode.Wait` so `TryWrite` returns `false` when full (the same Wait+TryWrite overflow-detection idiom the distributor documents at `SessionEventDistributor.cs:354-361`).
+2. **Fault immediately on a staging `TryWrite` miss.** A full staging channel means `EventWriteLoopAsync` has been saturated against a full `_events` for however long the worker took to emit `2 × capacity` further events — that *is* the sustained-slow-drain signal, with no timer needed. `StageWorkerEvent` treats a `false` return as `SetFaulted(ProtocolViolation, …)` unless the client is already terminal (a completed channel also returns `false` during shutdown — guard with the existing `IsTerminalState()` so the shutdown path stays a silent drop, as documented today). `SetFaulted` is non-blocking, so calling it from `DispatchEnvelope` on the read loop preserves the GWC-04 guarantee that the read loop never awaits behind events. Keep the existing `EventChannelFullModeTimeout` timed-write fault in `EnqueueWorkerEventAsync` unchanged — it catches the full-stall case earlier than the staging bound would.
+3. **Unify the queue-depth gauge over both channels.** Move the `Interlocked.Increment(ref _eventQueueDepth)` + `SetWorkerEventQueueDepth` from `EnqueueWorkerEventAsync` (`:616-617`, `:627-628`) to `StageWorkerEvent` (on successful `TryWrite`). The decrement already happens at consumer read (`ReadEventsCoreAsync:289`), so the single counter then reports *total undelivered events* (staged + queued) with no second counter and no extra hot-path work. Record `_metrics?.QueueOverflow("worker-event-staging")` on the staging fault so it is distinguishable from the existing `"worker-events"` overflow label.
+
+Rejected alternatives: (a) unbounded channel + a total-depth ceiling check — same effect but needs a second configured ceiling and a watermark comparison per event; the bounded channel gives the ceiling for free and fails at exactly the point the invariant is violated. (b) fault-on-sustained-growth (timestamp when the backlog first appears, fault when it persists past a window) — more machinery and a weaker guarantee, since memory still grows throughout the window. Fail-fast is the repo's documented backpressure policy (`docs/DesignDecisions.md`); a loud session fault with a worker kill is the correct outcome, not silent buffering.
+
+Coordinate with (do not block on) open GWC-21: if `EventChannelFullModeTimeout` is made configurable in the same pass, the 2× staging multiplier is the natural companion knob; until then the derived constant needs no validator change.
+
+**Implementation.**
+- `Workers/WorkerClient.cs` constructor (`:93-100`): replace `Channel.CreateUnbounded<WorkerEvent>` with `Channel.CreateBounded<WorkerEvent>(new BoundedChannelOptions(checked(2 * _options.EventChannelCapacity)) { SingleReader = true, SingleWriter = true, FullMode = BoundedChannelFullMode.Wait, AllowSynchronousContinuations = false })`. Rewrite the `_eventStaging` field comment (`:28-33`) — it must state the bound, the overflow fault, and that the depth gauge covers staged + queued.
+- `StageWorkerEvent` (`:565-573`): on `TryWrite` success, `Interlocked.Increment(ref _eventQueueDepth)` + `_metrics?.SetWorkerEventQueueDepth(...)`; on failure, `if (IsTerminalState()) return;` then `_metrics?.QueueOverflow("worker-event-staging")` and `SetFaulted(WorkerClientErrorCode.ProtocolViolation, …)` with a message naming the staging capacity (`2 × EventChannelCapacity`), the current depth, and the actionable fix (attach/unblock the `StreamEvents` consumer or raise `MxGateway:Events:QueueCapacity`). Update the XML doc, which currently asserts `TryWrite` always succeeds.
+- `EnqueueWorkerEventAsync` (`:610-647`): remove the two depth increments (now counted at staging); keep the timed `WriteAsync`, both fault paths, and the `Volatile.Read` depth in the overflow message.
+- `ReadEventsCoreAsync` (`:284-293`): unchanged — the decrement site is already correct for the unified counter.
+- Docs, same commit: `docs/MxAccessWorkerInstanceDesign.md` / `docs/GatewayProcessDesign.md` wherever the GWC-04 read-loop/event-writer decoupling is described (state the staging bound and the two fault modes); `docs/GatewayConfiguration.md` under `MxGateway:Events:QueueCapacity` (total per-session gateway buffering is 3× the value; overflow of either bound faults the session); the metrics doc if it defines the worker event queue-depth gauge (now includes staged events).
+- Tests (`Gateway/Workers/WorkerClientTests.cs`), new:
+  - `StagingChannelOverflowFaultsWorkerWithoutWaitingForFullModeTimeout` — small `EventChannelCapacity` (e.g. 4) via `WorkerClientOptions`, no event consumer attached, long `EventChannelFullModeTimeout` (e.g. 5 min so the timed-write fault cannot be the trigger); push `> 3 × capacity` events through the fake pipe; assert the client faults `ProtocolViolation` with the staging message, and that a command reply interleaved before the fault still completed (read loop never stalled).
+  - `WorkerEventQueueDepthGaugeCountsStagedEvents` — no consumer, push N events (N below the staging bound but above `EventChannelCapacity`); assert the gauge reports N (staged + queued); attach the consumer, drain, assert the gauge returns to 0.
+  - Existing GWC-04 tests (reply-not-stalled-behind-events, sustained-overflow fault) must stay green.
+
+**Verification.** `dotnet build src/ZB.MOM.WW.MxGateway.NonWindows.slnx`; `dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter "FullyQualifiedName~WorkerClientTests"`. Gateway-side only — no worker change, no windev/x86 needed.
+
+---
+
+## GWC-25 — Empty-ring ReplayGap sentinel carries `oldest_available_sequence = 0`, dead-streaming a compliant client   `Medium` · `P0`
+
+**Finding.** `RegisterWithReplay`'s empty-ring branch sets `oldestAvailableSequence = 0` even when `gap == true` (`Sessions/SessionEventDistributor.cs:463-467`), and `EventStreamService` emits that verbatim in the ReplayGap sentinel (`Grpc/EventStreamService.cs:133-139`, `:210-222`). The proto contract (`src/ZB.MOM.WW.MxGateway.Contracts/Protos/mxaccess_gateway.proto`, `ReplayGap.oldest_available_sequence`) instructs a client that wants a gapless resume to send `after_worker_sequence = oldest_available_sequence − 1`. With `oldest = 0`, a uint64 client computes `2^64−1`; on the next resume `RegisterWithReplay` replays nothing and reports no gap (the `afterSequence == ulong.MaxValue` wrap guard at `:471`/`:757` suppresses it), and the live filter `mxEvent.WorkerSequence <= afterWorkerSequence` (`EventStreamService.cs:179`) then drops **every** live event — a silently dead stream. Reachable by default: `ReplayRetentionSeconds = 300` age-evicts inside `RegisterWithReplay` via `EvictAged()` (`:458`), and `ReplayBufferCapacity = 0` with a behind cursor hits the same branch. All five clients shipped the `oldest − 1` formula (CLI-15).
+
+**Impact.** A client that reconnects after ≥ 5 minutes detached (or against a retention-disabled gateway) and follows the documented resume formula gets a permanently event-less stream with no error — the exact resilience feature (detach-grace + replay, on by default) fails in its headline scenario.
+
+**Design.** In the empty-ring-with-gap branch, emit `oldest_available_sequence = _highestSequenceSeen + 1` — the next sequence that can possibly be delivered. The client's `oldest − 1` then equals `_highestSequenceSeen`: the follow-up resume replays nothing, reports no gap, and the live filter passes every event newer than the highest already seen. Nothing is lost relative to today (the evicted events are unrecoverable either way — the sentinel's whole job is to say "re-snapshot"), and no client changes. Keep `0` for the no-gap case, where the field is documented as meaningless and never emitted. `TryGetReplayFrom` (`:735-774`) does not surface an oldest-available value, so it needs no change. Rejected alternative: define `0` as a special "nothing retained" case in the proto and handle it in all five clients — five client releases plus a version bump (CLI-39 is already blocking publishes) versus a one-line server fix that keeps the single resume formula universal.
+
+The proto comment currently states "`oldest_available_sequence` itself IS still retained", which becomes false in the empty-ring case — per the docs-with-source rule, amend the field comment in the same commit to define the empty-ring value ("when nothing is retained, this is the next sequence that can be delivered — `highest observed + 1` — and the `oldest − 1` resume formula remains valid; the interval evicted is unchanged"). This is a comment-only proto change (no descriptor delta), but the repo's codegen rules still apply — see the steps.
+
+**Implementation.**
+- `Sessions/SessionEventDistributor.cs:463-467`: replace `oldestAvailableSequence = 0;` with `oldestAvailableSequence = gap ? _highestSequenceSeen + 1 : 0;` plus a comment explaining the `oldest − 1` client formula this must keep valid (cite this finding).
+- `src/ZB.MOM.WW.MxGateway.Contracts/Protos/mxaccess_gateway.proto` (`ReplayGap.oldest_available_sequence`, ~line 759): append the empty-ring sentence above. Then regenerate per repo rules: delete `src/ZB.MOM.WW.MxGateway.Contracts/Generated/*.cs`, `dotnet build src/ZB.MOM.WW.MxGateway.Contracts/ZB.MOM.WW.MxGateway.Contracts.csproj`, and **commit `Generated/`** (net48 worker builds break otherwise). Sync the vendored client copies of the proto byte-identical (`clients/*/`); a comment-only edit changes no descriptor, so: Python `*_pb2*` output is unchanged (comments are not embedded — regenerate with the pinned grpcio-tools only if the files actually differ), Go/C#/Rust generated doc comments will churn — regenerate those per each client README, and revert spurious Java aggregate-file churn if no message-level delta appears (per the established Java convention).
+- `docs/Sessions.md` (~lines 228-234, ReplayGap section): document the empty-ring sentinel value and that `after_worker_sequence = oldest_available_sequence − 1` is the universal resume formula in both the retained and fully-evicted cases.
+- Tests, new:
+  - `Gateway/Sessions/SessionEventDistributorTests.cs` — `RegisterWithReplayReportsNextDeliverableSequenceWhenRingEmptiedByAge`: FakeTimeProvider, capacity > 0, short retention; pump events up to highest sequence H; advance the clock past retention; `RegisterWithReplay(afterSequence: 1, …)` → asserts `gap == true`, `oldestAvailableSequence == H + 1`, `replayedEvents` empty, `liveResumeSequence == 1`.
+  - `RegisterWithReplayReportsNextDeliverableSequenceWithRetentionDisabled`: capacity-0 internal ctor overload, events seen to H, resume behind → `gap == true`, `oldestAvailableSequence == H + 1`.
+  - `ResumeUsingSentinelFormulaAfterEmptyRingGapDeliversLiveEvents`: after the gap above, re-register with `afterSequence = (H + 1) − 1 = H` → no gap, no replay, and a newly pumped event `H + 1` is delivered on the live channel — the direct regression test for the dead-stream failure.
+  - `Gateway/GatewayEndToEndReconnectReplayTests.cs` — `ReconnectAfterFullAgeEvictionResumesWithSentinelFormula`: fake-worker e2e; stream, detach, evict everything by advancing time past `ReplayRetentionSeconds`, reconnect with the last-seen watermark, apply `oldest − 1` from the received sentinel on the follow-up resume, assert new live events arrive.
+
+Cross-domain: the client-side halves — Python CLI crashing on the sentinel (CLI-35) and the Go CLI destroying it (CLI-36) — are planned in the clients remediation doc; this server fix is independent and safe to land first (roadmap P0.1 groups all three).
+
+**Verification.** `dotnet build src/ZB.MOM.WW.MxGateway.NonWindows.slnx` (includes the contracts regen); `dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter "FullyQualifiedName~SessionEventDistributorTests"` and `--filter "FullyQualifiedName~GatewayEndToEndReconnectReplay"`. If the proto comment lands: the committed `Generated/` keeps the net48 worker buildable, but an actual x86 worker build/test run needs windev — flag as the standard windev verification (comment-only change, so functional risk is nil).
+
+---
+
+## GWC-26 — Alarm monitor attaches its subscriber after SubscribeAlarms; window transitions bypass the feed, missed Acknowledge never repaired   `Low` · `P2`
+
+**Finding.** `RunMonitorAsync` invokes `SubscribeAlarmsAsync` and the first `ReconcileAsync` (`Alarms/GatewayAlarmMonitor.cs:213-214`) *before* the internal distributor subscriber is attached inside `_sessionManager.ReadAlarmEventsAsync` (`:232-234` → `Sessions/SessionManager.cs:195-208` → `GatewaySession.AttachInternalEventSubscriber`, `Sessions/GatewaySession.cs:553-562`). The distributor pump has been running since `MarkReady` started the dashboard mirror (`GatewaySession.cs:445-449`, `:621`), and a late subscriber misses previously fanned events by design (`SessionEventDistributor.cs:579-582`). Raise/Clear lost in the window are repaired by the next reconcile, but `ApplyReconcile` broadcasts only presence deltas (`GatewayAlarmMonitor.cs:514-553`): an Acknowledge that lands in the window updates `_alarms` silently on the snapshot replace (`:547-551`) with no `Broadcast`, so live `StreamAlarms` subscribers show the alarm unacked until it clears.
+
+**Impact.** A one-shot window of a few hundred ms (two worker command round-trips) per monitor lifecycle in which alarm transitions reach only the dashboard mirror; a missed Acknowledge is never broadcast to alarm-feed subscribers. Same defect class as the fixed GWC-01, shrunk from permanent to transient.
+
+**Design.** Two independent halves:
+
+1. **Attach before subscribing.** The monitor already holds the `GatewaySession` instance returned by `OpenSessionAsync` (`:203-208`), so take the internal lease directly — `session.AttachInternalEventSubscriber()` — *before* `SubscribeAlarmsAsync`, and enumerate `lease.Reader.ReadAllAsync(...)` after reconcile. Transitions arriving during subscribe + reconcile simply buffer in the lease's bounded channel (`MxGateway:Events:QueueCapacity`, default 1024 — ample for a two-round-trip window); if it ever overflowed, the internal subscriber is merely disconnected (never faults the session, per the `isInternal` contract), the enumeration ends, and the monitor's supervisor loop restarts — acceptable and loud. Processing buffered transitions *after* `ApplyReconcile` is order-safe: `ApplyTransition` handles alarms already present in the reconciled snapshot. With the monitor calling the session directly, `ISessionManager.ReadAlarmEventsAsync` loses its only caller — remove it rather than leave dead public surface (mirrors the GWC-19 precedent). Rejected alternative: buffer inside `ReadAlarmEventsAsync` by splitting attach/enumerate on the manager — same effect, but keeps an indirection whose only job was to hide the session object the monitor already has.
+2. **Repair acked-state in reconcile.** In `ApplyReconcile`, for references present in both `_alarms` and the incoming snapshot, broadcast an `Acknowledge` feed transition when the state changed to acked (`existing.CurrentState != incoming.CurrentState && incoming.CurrentState == AlarmConditionState.ActiveAcked`), via the existing `TransitionFromSnapshot(incoming, AlarmTransitionKind.Acknowledge)`. This is defense-in-depth for any future missed window (worker restart, overflow disconnect), not just this one. Invariant note: `ApplyReconcile` already broadcasts snapshot-derived Raise/Clear transitions on the **alarm feed** (`AlarmFeedMessage`, the StreamAlarms/dashboard surface — see the method comment "broadcasting a synthetic transition for any alarm the live stream missed"); extending that established feed-level repair to acked-state deltas does not touch the gRPC `StreamEvents` path and therefore does not violate the "never synthesize events" rule, which governs `MxEvent` emission.
+
+**Implementation.**
+- `Alarms/GatewayAlarmMonitor.cs` `RunMonitorAsync` (`:211-250`): after `OpenSessionAsync`/`_session` publication, `using IEventSubscriberLease alarmLease = session.AttachInternalEventSubscriber();` **before** `SubscribeAlarmsAsync(...)`; replace the `_sessionManager.ReadAlarmEventsAsync(...)` enumeration with `await foreach (MxEvent mxEvent in alarmLease.Reader.ReadAllAsync(linked.Token))`; update the `:228-231` comment to state the attach-before-subscribe ordering and why.
+- `Sessions/SessionManager.cs:195-208` and `ISessionManager`: remove `ReadAlarmEventsAsync` (re-verify zero remaining callers with a grep at implementation time; repoint any test that used it at `GatewaySession.AttachInternalEventSubscriber`).
+- `Alarms/GatewayAlarmMonitor.cs` `ApplyReconcile` (`:514-553`): before the snapshot replace, add the both-present acked-delta loop described above.
+- Docs, same commit: `docs/Sessions.md`'s note that the alarm monitor attaches as an internal distributor subscriber (added by the GWC-01 fix) — extend with "attached before SubscribeAlarms so no transition window is missed"; `gateway.md`'s alarm-monitor section if it describes the startup ordering.
+- Tests:
+  - `GatewayAlarmMonitorAttachOrderTests` (new, or extend the existing alarm monitor tests in `Alarms/`): fake worker emits an `Acknowledge` (and a `Raise`) transition immediately after the `SubscribeAlarms` reply and before the first reconcile reply; assert a `StreamAlarms` subscriber observes both — the direct regression test for the window.
+  - `ApplyReconcileBroadcastsAcknowledgeDelta` (unit): seed the cache with an `Active` alarm, reconcile with the same reference `ActiveAcked`; assert exactly one `Acknowledge` feed message and no Raise/Clear.
+  - Existing alarm monitor + `SessionManagerTests` suites stay green after the `ReadAlarmEventsAsync` removal.
+
+**Verification.** `dotnet build src/ZB.MOM.WW.MxGateway.NonWindows.slnx`; `dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter "FullyQualifiedName~GatewayAlarmMonitor"` and `--filter "FullyQualifiedName~SessionManagerTests"`. Depends on GWC-27 landing first (or together) so the earlier attach point runs against the readiness gate.
+
+---
+
+## GWC-27 — `AttachInternalEventSubscriber` bypasses the readiness gate; premature attach poisons the distributor permanently   `Low` · `P2`
+
+**Finding.** `GatewaySession.AttachInternalEventSubscriber` (`Sessions/GatewaySession.cs:553-562`) runs `EnsureDistributorCreated` → `Register(isInternal: true)` → `StartPumpIfRequested` with no state check, unlike `AttachEventSubscriber`, which requires `_state == Ready && _workerClient?.State == Ready` under `_syncRoot` (`:889-896`). `SessionManager.ReadAlarmEventsAsync` (`Sessions/SessionManager.cs:195-208`) exposes the unchecked path publicly. If invoked before Ready, the pump's source (`MapWorkerEventsAsync` → `GetReadyWorkerClientAsync`, `:740-749`, `:1910`) throws `SessionNotReady`; `PumpAsync` completes all subscribers with the error and latches `_completed` (`SessionEventDistributor.cs:603-612`, `:662-680`). `_eventDistributorStarted` is never reset (`GatewaySession.cs:527-531`) and the distributor is replaced only in `DisposeAsync`, so every later registration — including gRPC `StreamEvents` — receives an immediately-completed channel carrying the stale error: a Ready session with permanently dead event streaming. Unreachable today (the alarm monitor attaches only after `OpenSessionAsync` returns Ready), but one future call site away and failing in the worst way (silent, permanent, per-session).
+
+**Impact.** Latent lifecycle hazard: any refactor or new caller that attaches internally before Ready silently kills event streaming for that session's whole lifetime, with no fault and no log pointing at the cause.
+
+**Design.** Mirror `AttachEventSubscriber`'s readiness gate at the top of `AttachInternalEventSubscriber`: under `_syncRoot`, throw `SessionManagerException(SessionManagerErrorCode.SessionNotReady, …)` when `_state != SessionState.Ready || _workerClient?.State != WorkerClientState.Ready`, *before* `EnsureDistributorCreated` — so a premature attach fails loudly and never constructs/starts the distributor. Rejected alternative: make the distributor resettable after a pump fault — a much larger lifecycle change (replay ring, `_completed` latch, dashboard mirror lease all assume one distributor per session) that would also mask caller bugs instead of surfacing them. `StartDashboardMirror` keeps its direct `distributor.Register(isInternal: true)` path — it is called from `MarkReady` under its own state guard (`:604`) and is not a public entry point.
+
+**Implementation.**
+- `Sessions/GatewaySession.cs:553-562`: add the `_syncRoot` readiness check (reuse the exact exception code and message shape from `AttachEventSubscriber:891-896`); then proceed with the existing `EnsureDistributorCreated`/`Register`/`StartPumpIfRequested` sequence outside the lock (matching `AttachEventSubscriber`'s lock discipline — the check is under the lock, the distributor calls are not). Extend the XML remarks to state the gate.
+- `Sessions/SessionManager.cs:195-208`: no separate check needed — the path inherits the gate via the session call, and GWC-26 removes the method entirely.
+- Tests (`Gateway/Sessions/GatewaySessionTests.cs`), new `AttachInternalEventSubscriberBeforeReadyThrowsAndDoesNotPoisonDistributor`: on a not-yet-Ready session, assert `SessionManagerException`/`SessionNotReady`; then drive the session to Ready and assert `AttachEventSubscriber` still yields a live stream (the key assertion: the distributor was never created/started by the failed attach).
+
+**Verification.** `dotnet build src/ZB.MOM.WW.MxGateway.NonWindows.slnx`; `dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter "FullyQualifiedName~GatewaySessionTests"`. Land before (or with) GWC-26.
+
+---
+
+## GWC-28 — Gateway→worker envelope `sequence` stamped at creation, not at write   `Low` · `P2`
+
+**Finding.** `CreateEnvelope` stamps `Sequence = (ulong)Interlocked.Increment(ref _nextSequence)` at envelope construction (`Workers/WorkerClient.cs:1031-1044`); envelopes are then enqueued to `_outboundEnvelopes` (`EnqueueAsync:957-972`) and written by the single `WriteLoopAsync` (`:390-409`); `WorkerFrameWriter` does not re-stamp. Two concurrent `InvokeAsync` callers can stamp 1 and 2 but enqueue in the order 2, 1 — non-monotonic sequences on the pipe, violating `gateway.md`'s "monotonic per sender" contract. This is exactly what WRK-04 fixed on the worker side (stamp at the point of writing, under the writer's serialization); the gateway half was not given the same treatment. Benign today — neither side validates inbound sequence (GWC-10/IPC-10 open) — but it would start faulting sessions the moment worker-side validation is added.
+
+**Impact.** Contract violation lying in wait: adding the specified sequence enforcement on the worker side (the mirror of GWC-10) would immediately fault healthy sessions under concurrent command load.
+
+**Design.** Stamp at write, matching WRK-04: `WriteLoopAsync` assigns `envelope.Sequence` immediately before `_writer.WriteAsync`. The write loop is the channel's single consumer (`SingleReader = true`, `:74`), so writes are naturally serialized — the counter becomes a plain `ulong` incremented only on the write loop, and the `Interlocked` goes away. Every outbound envelope flows through `_outboundEnvelopes` (hello `:150`, commands `:223`, shutdown `:317`), so coverage is total. Rejected alternative: re-stamp inside `WorkerFrameWriter` — the frame writer is a pure framing layer shared with the `FakeWorkerHarness` and tests; giving it protocol state would leak envelope semantics into the wire layer on both sides.
+
+**Implementation.**
+- `Workers/WorkerClient.cs:1031-1044` (`CreateEnvelope`): remove the `Sequence` assignment; add a comment that the sequence is stamped in `WriteLoopAsync` at the point of writing (cite WRK-04 parity and this finding).
+- `Workers/WorkerClient.cs:390-409` (`WriteLoopAsync`): before `_writer.WriteAsync(envelope, …)`, `envelope.Sequence = ++_nextSequence;` — change the `_nextSequence` field (`:37`) from `long` + `Interlocked` to a plain `ulong` touched only by the write loop, and update its usage accordingly.
+- Docs, same commit: the `gateway.md` envelope-sequence paragraph (the "monotonic per sender" contract, ~line 314) — note both sides stamp at write under their single write path; cross-reference open GWC-10 for the (still absent) inbound enforcement.
+- Tests (`Gateway/Workers/WorkerClientTests.cs`), new `ConcurrentInvokesEmitStrictlyIncreasingSequencesOnTheWire`: fake pipe; fire N (e.g. 32) parallel `InvokeAsync` calls; capture the frames the "worker" side receives; assert the `Sequence` values are strictly increasing in wire order. Existing handshake/shutdown tests confirm hello/shutdown envelopes still carry sequences.
+
+**Verification.** `dotnet build src/ZB.MOM.WW.MxGateway.NonWindows.slnx`; `dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter "FullyQualifiedName~WorkerClientTests"` and `--filter "FullyQualifiedName~WorkerFrameProtocolTests"`. Gateway-side only; the worker's own WRK-04 stamping is untouched.
+
+---
+
+## GWC-29 — `Invoke` deep-clones the entire request only to discard the cloned command   `Low` · `—`
+
+**Finding.** `MxAccessGatewayService.Invoke` runs `MxCommandRequest invokeRequest = request.Clone(); invokeRequest.Command = commandToInvoke;` (`Grpc/MxAccessGatewayService.cs:119-120`) — a deep clone of the whole request *including its command payload* (potentially a large bulk-write graph), whose cloned command is immediately overwritten and discarded. `MapCommand` then performs the one clone actually needed (`request.Command.Clone()`, `Grpc/MxAccessGrpcMapper.cs:27-37`). Net cost: one full wasted command deep-clone per `Invoke` — the same cost class GWC-07/IPC-05 eliminated on the event and envelope paths.
+
+**Impact.** Avoidable allocation and copy proportional to command size on every command, worst for exactly the bulk writes that are largest.
+
+**Design.** Skip the request clone entirely: add a `MapCommand(MxCommand command)` overload that builds the `WorkerCommand` from the command alone (`Command = command.Clone()`, `EnqueueTimestamp` as today) — `MapCommand` reads nothing else from the request. `Invoke` passes `commandToInvoke` directly. Keep the existing `MapCommand(MxCommandRequest)` overload delegating to the new one so other callers and tests are untouched. The single remaining clone is still required and must stay: `commandToInvoke` may be the caller-owned `request.Command` (gRPC-owned graph) and is also read *after* the invoke by `session.TrackCommandReply(commandToInvoke, …)` (`MxAccessGatewayService.cs:132`), so ownership transfer (à la GWC-07) is not safe here — the clone in `MapCommand` is what keeps `WorkerClient.CreateCommandEnvelope`'s documented no-aliasing invariant (`Workers/WorkerClient.cs:1000-1004`) true. That comment stays accurate unchanged.
+
+**Implementation.**
+- `Grpc/MxAccessGrpcMapper.cs`: add `public WorkerCommand MapCommand(MxCommand command)` (null-check, clone, timestamp); retarget `MapCommand(MxCommandRequest request)` to validate and delegate.
+- `Grpc/MxAccessGatewayService.cs:119-121`: delete the `request.Clone()` and the `invokeRequest` local; `WorkerCommand workerCommand = mapper.MapCommand(commandToInvoke);`.
+- Tests: `Gateway/Grpc/MxAccessGrpcMapperTests.cs` — new `MapCommandFromCommandClonesPayload` (mutating the returned `WorkerCommand.Command` does not affect the input command; `EnqueueTimestamp` populated; both overloads produce equal results). Existing `MxAccessGatewayServiceTests` and `MxAccessGatewayServiceConstraintTests` must stay green — the constraint path (`bulkConstraintPlan.Command` flowing through, denied-merge on the reply) is the behavior most worth re-running.
+- Docs: none (internal hot-path change; no contract or config surface).
+
+**Verification.** `dotnet build src/ZB.MOM.WW.MxGateway.NonWindows.slnx`; `dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter "FullyQualifiedName~MxAccessGrpcMapperTests"` and `--filter "FullyQualifiedName~MxAccessGatewayService"`.
+
+---
+
+## GWC-30 — Frame reader allocates a fresh 4-byte length-prefix array per frame   `Info` · `—`
+
+**Finding.** `WorkerFrameReader.ReadAsync` allocates `new byte[sizeof(uint)]` per frame (`Workers/WorkerFrameReader.cs:33`); the GWC-08 pass pooled the payload buffer but left the prefix. Gen-0, 4 bytes — negligible; recorded so the next hot-path pass knows it was seen.
+
+**Impact.** One tiny short-lived allocation per inbound frame. No functional risk.
+
+**Design.** A reusable per-reader scratch field: `private readonly byte[] _lengthPrefix = new byte[sizeof(uint)];`. The reader is single-consumer by construction — exactly one read loop per `WorkerClient` calls `ReadAsync`, never concurrently (handshake reads happen before the read loop starts) — so a per-instance buffer is safe; document that `ReadAsync` is not reentrant. Rejected: pooling 4 bytes via `ArrayPool` — rent/return overhead exceeds the allocation it saves.
+
+**Implementation.**
+- `Workers/WorkerFrameReader.cs`: add the `_lengthPrefix` field, use it at `:33-36`, and add a one-line class remark that `ReadAsync` must not be invoked concurrently (single-consumer contract).
+- Tests: existing `Gateway/Workers/WorkerFrameProtocolTests.cs` round-trips cover it; add (if absent) a sequential multi-frame read on one reader instance asserting all frames parse — guarding the shared-scratch reuse.
+- Docs: none.
+
+**Verification.** `dotnet build src/ZB.MOM.WW.MxGateway.NonWindows.slnx`; `dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter "FullyQualifiedName~WorkerFrameProtocolTests"`.

archreview/2026-07-12/remediation/20-worker.md

@@ -0,0 +1,539 @@
+# Worker Process — Remediation Design & Implementation (2026-07-12 cycle)
+
+Source review: [20-worker.md](../20-worker.md) · Roadmap: [00-overall.md](../00-overall.md) · Generated: 2026-07-13 · Baseline: `main` @ `4f5371f`
+
+This cycle's findings are concentrated in the new frame-write scheduler and the DrainEvents bound
+that the prior remediation added — the scheduler is fundamentally sound, and every fix below is an
+edge-hardening of it rather than a redesign. One finding is P0 (WRK-21, shared with IPC-23: a
+diagnostics command can still kill the session); the rest are Lows plus two residuals of prior Done
+items (WRK-25, WRK-26). All fixes preserve MXAccess parity (no MXAccess behavior changes — these
+are all IPC/liveness-layer changes), preserve the control-before-event write-priority guarantee,
+and use net48-safe constructs only (plain constructors and get-only properties, no init-only
+members, no positional records). The worker builds and tests only on the Windows x86 host
+(windev); verification commands assume that host.
+
+## Finding register
+
+| ID | Sev | Tier | Eff | Dep | Status | Title |
+|----|-----|------|-----|-----|--------|-------|
+| WRK-21 | Medium | P0 | M | IPC-23 (same defect, fix owned here); WRK-28 (same lines) | Not started | DrainEvents bound is count-based only; an oversized reply still kills the session and loses the drained events |
+| WRK-22 | Low | — | S | IPC-26 (same defect, fix owned here) | Not started | Cancelled `WriteAsync` leaves its frame queued; it is still written later |
+| WRK-23 | Low | — | S | WRK-21 (rejection path becomes backstop-only) | Not started | Rejected frames consume sequence numbers, producing wire gaps |
+| WRK-24 | Low | — | S | — | Not started | `AdoptNegotiatedMaxMessageBytes` has no lower-bound sanity check |
+| WRK-25 | Low | P2 | S | WRK-22 (both touch enqueue/dequeue) | Not started | WRK-12 flush coalescing never engages on the event hot path |
+| WRK-26 | Low | P1 | S | WRK-23 (soft — sequence prose); discharges IPC-29 | Not started | Write-priority and overflow doc drift from the WRK-07 change |
+| WRK-27 | Low | — | S | — | Not started | Alarm poll bypasses the watchdog's in-flight suppression (15 s vs 75 s) |
+| WRK-28 | Low | — | S | WRK-21 (land in the same commit cluster) | Not started | 10,000 drain cap is a duplicated magic constant with a comment-only sync contract |
+
+---
+
+## WRK-21 — DrainEvents bound is count-based only; an oversized reply still kills the session and loses the drained events   `Medium` · `P0`
+
+**Finding.** `CreateDrainEventsReply` clamps only the event *count* to `MaxDrainEventsPerReply`
+(10,000) (`src/ZB.MOM.WW.MxGateway.Worker/Ipc/WorkerPipeSession.cs:545-557`); the reply is written
+via `WriteControlReplyAsync` (:481, :485-496) with no catch. A reply whose serialized size exceeds
+the negotiated `MaxMessageBytes` throws `WorkerFrameProtocolException(MessageTooLarge)` from
+`WorkerFrameWriter.WriteFrameAsync` (`Ipc/WorkerFrameWriter.cs:250-255`) — correctly a *per-frame*
+rejection at the writer, but `HandleControlCommandAsync`/`DispatchGatewayEnvelopeAsync` propagate
+it out of `RunMessageLoopAsync`, so the session dies and `WorkerApplication.cs:110-119` exits
+`ProtocolViolation` (6). The 10,000 drained events were already removed from `MxAccessEventQueue`
+by `runtimeSession.DrainEvents(maxEvents)` (:551) and are gone. 10,000 events at ~1.7 KiB each
+(large string/array `MxValue`s) exceed the default 16 MiB + 64 KiB frame max, so the comment's
+stated goal ("cannot pack the whole queue into one session-killing reply frame", :21-26) is not met
+for byte-heavy events. **This is the same defect as IPC-23; this entry owns the single fix** (the
+IPC remediation register should point here). Roadmap P0 item 2: byte-aware DrainEvents reply
+capping such that **no diagnostics command can be session-fatal**.
+
+**Impact.** A `DrainEvents max_events=0` diagnostic against a byte-heavy queue — the array-heavy
+payloads this gateway exists for — terminates a healthy session (exit 6, gateway kills the worker)
+*and* silently destroys the drained events. The exact "session-killing reply frame" the prior fix
+(IPC-04/WRK slice) was added to prevent, narrowed but not closed.
+
+**Design.** Two layers, both required by the P0 acceptance criterion:
+
+1. **Primary — byte-aware drain that never removes an event it cannot ship.** The reply must be
+   sized *while draining*, and the size decision must happen inside the queue's lock so events are
+   only dequeued once they are known to fit. Add a byte-budgeted overload to `MxAccessEventQueue`:
+   drain from the head while `count < maxEvents` **and** the accumulated serialized size plus the
+   next event's size stays within a byte budget; an event that does not fit is left at the head for
+   the next call. The budget is `_options.MaxMessageBytes` (the negotiated value, read at
+   reply-build time) minus a fixed headroom constant covering the envelope/reply wrapper overhead
+   (64 KiB — the same envelope-overhead reserve rationale `docs/WorkerFrameProtocol.md` already
+   documents for the frame max itself). Per-event cost is `WorkerEvent.CalculateSize() + 8` — the
+   `WorkerEvent` wrapper slightly overestimates the packed `MxEvent`, and +8 conservatively covers
+   the repeated-field tag and length varint, so the estimate errs strictly on the safe side.
+   Truncation is reported in the reply's existing `DiagnosticMessage` field ("N events returned,
+   M remain; repeat DrainEvents for the rest"), so **no proto change** is needed.
+
+   Degenerate case: a single head event whose serialized size alone exceeds the budget. It is
+   **not** drained (draining it would either build an oversized reply or lose it); the reply
+   returns whatever fit before it (possibly zero events) with a `DiagnosticMessage` naming the
+   blocked event's `WorkerSequence` so the operator can find the offending tag. Such an event
+   cannot be shipped on the streaming path either — that policy question is IPC-30 and stays out
+   of scope here; the WRK-21 guarantee is only that DrainEvents never dies and never loses events.
+
+2. **Backstop — the control-reply write seam must not be session-fatal.** Even with capping (a
+   future command, a sizing bug), catch `WorkerFrameProtocolException` with
+   `ErrorCode == MessageTooLarge` at the two reply-write seams and answer the correlation with a
+   small error reply instead of unwinding the session: `HandleControlCommandAsync`'s
+   `WriteControlReplyAsync` call (:481) and `ProcessCommandAsync`'s reply write (:630-638), which
+   today converts *any* write failure into `_state = Faulted` + `MxaccessCommandFailed` — i.e. an
+   oversized STA command reply also faults the whole session. The error reply uses
+   `ProtocolStatusCode.InvalidRequest` with an explanatory message ("reply exceeded the negotiated
+   frame maximum; retry with a smaller request").
+
+Rejected alternatives: (a) adding `truncated`/`remaining_count` fields to `DrainEventsReply` — a
+contracts change that regenerates and ripples through all five language clients for a diagnostic
+command; `DiagnosticMessage` carries the same information at zero contract cost. (b) Catch-only
+(no byte capping) — survives the session but still loses the already-drained events, failing the
+"no event loss" half. (c) Draining into the reply and measuring `reply.CalculateSize()`
+periodically — protobuf C# does not memoize sizes, making that O(n²), and events over the line
+would already be dequeued. (d) A new `ProtocolStatusCode.ReplyTooLarge` enum value — proto change
+for a should-be-unreachable backstop; rejected for the same ripple reason as (a).
+
+Parity note: DrainEvents is a gateway-protocol diagnostic, not an MXAccess call — no MXAccess
+behavior changes. Control-before-event priority is untouched (the reply stays a Control frame).
+
+**Implementation.**
+1. `src/ZB.MOM.WW.MxGateway.Worker/MxAccess/WorkerEventDrainResult.cs` (new): sealed class, plain
+   constructor + get-only properties (net48-safe): `IReadOnlyList<WorkerEvent> Events`,
+   `bool TruncatedBySize`, `int RemainingCount`, `ulong OversizedHeadSequence` (0 = none).
+2. `MxAccess/MxAccessEventQueue.cs`: add
+   `public WorkerEventDrainResult Drain(uint maxEvents, int maxTotalBytes)` — under `syncRoot`,
+   loop `events.Peek()`, compute `workerEvent.CalculateSize() + 8`, dequeue while it fits the
+   remaining budget and `drained.Count < maxEvents` (0 = existing "all" semantics for the count
+   dimension); on a head event that alone exceeds `maxTotalBytes`, stop and record its
+   `WorkerSequence` into `OversizedHeadSequence`; `RemainingCount = events.Count` on exit.
+3. `MxAccess/IWorkerRuntimeSession.cs`: add
+   `WorkerEventDrainResult DrainEvents(uint maxEvents, int maxTotalBytes);` (no default interface
+   method — the worker is net48, which has no DIM runtime support). Implement in
+   `MxAccess/MxAccessStaSession.cs` (delegate to the queue) and in every test fake that implements
+   the interface (the fake runtime session in `Worker.Tests/Ipc/WorkerPipeSessionTests.cs`).
+4. `Ipc/WorkerPipeSession.cs`:
+   - add `private const int DrainReplyFrameHeadroomBytes = 64 * 1024;` next to
+     `MaxDrainEventsPerReply` (:26) with a comment tying it to the envelope-overhead reserve;
+   - `CreateDrainEventsReply` (:537-562): call the byte-budgeted overload with
+     `_options.MaxMessageBytes - DrainReplyFrameHeadroomBytes`; when `TruncatedBySize`, set
+     `reply.DiagnosticMessage` with returned/remaining counts and, when
+     `OversizedHeadSequence != 0`, the blocked sequence;
+   - `HandleControlCommandAsync` (:481): wrap `WriteControlReplyAsync(reply, ...)` in
+     `try/catch (WorkerFrameProtocolException ex) when (ex.ErrorCode == WorkerFrameProtocolErrorCode.MessageTooLarge)`
+     → log `WorkerControlReplyTooLarge` (correlation id, kind, payload size from the exception
+     message) and write a fallback `MxCommandReply` (same correlation id/kind,
+     `ProtocolStatusCode.InvalidRequest`, explanatory message) via `WriteControlReplyAsync`;
+   - `ProcessCommandAsync` (:630-638): put the same catch around the reply write specifically
+     (inner try), writing the fallback error reply for the correlation instead of falling into the
+     generic catch that sets `_state = WorkerState.Faulted`.
+5. Fold WRK-28 into this commit cluster (same lines — see below).
+6. Docs same commit: `docs/MxAccessWorkerInstanceDesign.md` — the control-command/DrainEvents
+   prose gains the byte cap, the truncation `DiagnosticMessage` contract, and the
+   oversized-head-event behavior; note in the Outbound Queues section that no control reply is
+   session-fatal on size. `docs/WorkerFrameProtocol.md` — one sentence in the size-limits section:
+   the session pre-sizes DrainEvents replies below the negotiated max, and `MessageTooLarge` on a
+   reply write is answered with an error reply, not session teardown.
+
+**Verification.** On windev:
+`dotnet build src/ZB.MOM.WW.MxGateway.Worker/ZB.MOM.WW.MxGateway.Worker.csproj -p:Platform=x86`
+then
+`dotnet test src/ZB.MOM.WW.MxGateway.Worker.Tests/ZB.MOM.WW.MxGateway.Worker.Tests.csproj -p:Platform=x86 --filter "FullyQualifiedName~WorkerPipeSessionTests|FullyQualifiedName~MxAccessEventQueueTests"`.
+New tests:
+- `WorkerPipeSessionTests.DrainEvents_ByteHeavyQueue_ReplyIsBoundedAndSessionSurvives` — the
+  review's named repro: enqueue 10,000 events carrying large string values (≥ ~1.7 KiB each) into
+  a real `MxAccessEventQueue` behind the fake runtime, send `DrainEvents max_events=0`; assert the
+  reply envelope's serialized size ≤ `MaxMessageBytes`, `DiagnosticMessage` reports truncation,
+  and the session still answers a subsequent `Ping` (no fault frame, `RunAsync` still running).
+- `WorkerPipeSessionTests.DrainEvents_RepeatedCalls_RecoverAllEventsWithoutLoss` — issue
+  DrainEvents until empty; assert the union of replies is exactly the 10,000 enqueued events in
+  order (split/bounded replies, zero loss).
+- `WorkerPipeSessionTests.ControlReplyTooLarge_WritesErrorReplyInsteadOfDying` — shrink the
+  negotiated max via `WorkerFrameProtocolOptions` so the reply write throws `MessageTooLarge`;
+  assert a fallback `InvalidRequest` reply with the same correlation id is written and the session
+  survives.
+- `WorkerPipeSessionTests.CommandReplyTooLarge_WritesErrorReplyInsteadOfFaulting` — same via the
+  STA command path (`ProcessCommandAsync`); assert no `WorkerFault`, `_state` stays `Ready`.
+- `MxAccessEventQueueTests.Drain_ByteBudget_StopsBeforeBudgetAndLeavesRemainderQueued` — order
+  preserved, `RemainingCount` exact, undrained events still present.
+- `MxAccessEventQueueTests.Drain_ByteBudget_OversizedHead_DrainsNothingAndReportsHeadSequence`.
+
+---
+
+## WRK-22 — Cancelled `WriteAsync` leaves its frame queued; it is still written later   `Low` · `—`
+
+**Finding.** `WorkerFrameWriter.WriteAsync` enqueues the `PendingFrame` before the cancellable
+lock wait (`Ipc/WorkerFrameWriter.cs:87-103`); if `_writeLock.WaitAsync(cancellationToken)`
+throws, nothing removes the frame, so the next lock-holder writes it. Concrete scenario: at
+shutdown the event-drain loop's token is cancelled (`Ipc/WorkerPipeSession.cs:298-304`) while its
+event frame waits for the lock; the shutdown-ack write then drains queues — ack first (Control),
+then the orphaned event — so an event frame is emitted *after* `WorkerShutdownAck`, and a write
+reported as cancelled to its caller still reaches the wire. **Same defect as IPC-26** (the IPC
+review describes this writer); this entry owns the single fix.
+
+**Impact.** Benign today (the gateway drops post-shutdown/unknown frames), but the "cancelled
+means not written" contract is silently violated, an unobserved completion lingers, and any future
+gateway-side strictness about frames after the ack turns this into a protocol violation.
+
+**Design.** Tombstone on cancellation, claim on dequeue. `Queue<T>` has no random removal, so the
+standard fix is: the cancelled caller marks its frame cancelled under `_gate`; `DequeueNext` skips
+completed (cancelled) frames and atomically marks the frame it returns as *claimed*. A frame
+already claimed by a draining lock-holder is mid-write and can no longer be recalled — the
+documented contract becomes "cancellation guarantees the frame is not written *if it has not yet
+been claimed*; a claimed frame may still reach the wire." That residual window is unavoidable
+without blocking the canceller behind the very write it is abandoning (rejected: it inverts the
+point of cancellation). Rejected alternative: switching to a `LinkedList` for O(1) removal —
+tombstoning is simpler, allocation-free, and the queues are short-lived.
+
+**Implementation.**
+1. `Ipc/WorkerFrameWriter.cs`, nested `PendingFrame`: add `public bool Claimed;` (plain field,
+   mutated only under `_gate`).
+2. `WriteAsync(envelope, priority, ct)` (:100-113): wrap `await _writeLock.WaitAsync(cancellationToken)`
+   in `try/catch (OperationCanceledException)` → `lock (_gate) { if (!frame.Claimed) { frame.Completion.TrySetCanceled(cancellationToken); } }`
+   then rethrow. (If already claimed, the frame will be written; the caller still observes the
+   cancellation — documented in the method's XML doc.)
+3. `DequeueNext` (:200-216): under `_gate`, loop-dequeue past frames whose
+   `Completion.Task.IsCanceled`; set `Claimed = true` on the frame actually returned.
+   `FailAllQueued`/`FailFrames` already use `TrySetException`, which no-ops on cancelled frames.
+4. Docs same commit: the "Write scheduling and sequencing" section of
+   `docs/WorkerFrameProtocol.md` (created by WRK-26 — if WRK-26 has not landed yet, carry the
+   cancellation paragraph in this commit and let WRK-26 absorb it) states the
+   cancelled-means-not-written-unless-claimed contract.
+
+**Verification.** On windev:
+`dotnet test src/ZB.MOM.WW.MxGateway.Worker.Tests/ZB.MOM.WW.MxGateway.Worker.Tests.csproj -p:Platform=x86 --filter "FullyQualifiedName~WorkerFrameProtocolTests"`.
+New tests:
+- `WorkerFrameProtocolTests.WriteAsync_CancelledWhileWaitingForLock_FrameIsNeverWritten` — block
+  the stream (gated fake stream) so writer A holds `_writeLock` mid-write; enqueue an event write
+  with a CTS; cancel; assert `OperationCanceledException`; unblock A; write a control frame;
+  assert the captured wire bytes contain A's frame and the control frame only — the cancelled
+  event envelope never appears.
+- `WorkerFrameProtocolTests.WriteAsync_CancelledEventFrame_DoesNotTrailShutdownAck` — the review's
+  shutdown scenario: cancelled event frame queued, then a Control (ack-shaped) write drains;
+  assert the event frame is skipped and the ack is the last frame on the wire.
+
+---
+
+## WRK-23 — Rejected frames consume sequence numbers, producing wire gaps   `Low` · `—`
+
+**Finding.** `WorkerFrameWriter.WriteFrameAsync` stamps `envelope.Sequence = unchecked(++_nextSequence)`
+(`Ipc/WorkerFrameWriter.cs:240`) *before* the empty-payload and `MessageTooLarge` checks
+(:243-255), so a per-frame rejection leaves a hole in the on-wire sequence. `gateway.md:328` calls
+`sequence` a monotonic *diagnostic* counter, so nothing breaks — but the existing test
+`WorkerFrameProtocolTests.WriteAsync_UnderConcurrentCalls_StampsGapFreeMonotonicSequence`
+documents a stronger gap-free expectation than the code guarantees once a rejection occurs.
+
+**Impact.** Diagnostics only: an operator correlating wire captures sees phantom gaps after any
+rejection and may chase a lost-frame bug that does not exist; the test asserts a guarantee the
+code does not hold.
+
+**Design.** Peek-stamp, validate, commit. The sequence value participates in `CalculateSize()`
+(varint width), so it cannot simply be stamped after the size checks — the checked size would not
+match the written payload. Instead compute a *candidate* (`_nextSequence + 1`), stamp it, run the
+size/empty checks against the stamped envelope, and only commit `_nextSequence = candidate`
+immediately before the stream write. On a per-frame rejection the counter is untouched, so the
+next accepted frame reuses the number and the wire stays contiguous. Safe because `_nextSequence`
+is only ever touched by the lock-holder (`:44-48`). The rejected envelope retains its tentative
+sequence — a caller-visible mutation on a failed envelope that already exists today. Rejected
+alternative: decrement on rejection (`--_nextSequence`) — equivalent effect but leaves a window
+where the counter was observably advanced; the candidate/commit form has no intermediate state.
+Dependency note: after WRK-21 lands, `MessageTooLarge` at the writer becomes a rare backstop, so
+this fix is what makes the existing gap-free test's guarantee genuinely true rather than
+usually-true.
+
+**Implementation.**
+1. `Ipc/WorkerFrameWriter.cs`, `WriteFrameAsync` (:234-267): keep
+   `WorkerEnvelopeValidator.Validate` first; replace the stamp with
+   `ulong candidate = unchecked(_nextSequence + 1); envelope.Sequence = candidate;`; after the
+   empty-payload and `MaxMessageBytes` checks pass, `_nextSequence = candidate;` immediately
+   before the buffer build/`WriteTo`. Update the :45-48 comment.
+2. Docs same commit: none required beyond WRK-26's scheduling section, which should state
+   "rejected frames do not consume sequence numbers" once this lands (see WRK-26 ordering note).
+
+**Verification.** On windev:
+`dotnet test src/ZB.MOM.WW.MxGateway.Worker.Tests/ZB.MOM.WW.MxGateway.Worker.Tests.csproj -p:Platform=x86 --filter "FullyQualifiedName~WorkerFrameProtocolTests"`.
+New test:
+- `WorkerFrameProtocolTests.WriteAsync_PerFrameRejection_DoesNotConsumeSequence` — write an
+  accepted frame, attempt an oversized frame (assert `MessageTooLarge` rejection), write another
+  accepted frame; assert the two on-wire frames carry sequences 1 and 2.
+The existing `WriteAsync_UnderConcurrentCalls_StampsGapFreeMonotonicSequence` must still pass.
+
+---
+
+## WRK-24 — `AdoptNegotiatedMaxMessageBytes` has no lower-bound sanity check   `Low` · `—`
+
+**Finding.** `Ipc/WorkerFrameProtocolOptions.cs:124-140` — 0 keeps the default and >256 MiB
+throws, but any value from 1 byte up is adopted. A gateway bug or a foreign/old gateway sending
+e.g. 512 leaves the session alive but unable to write any frame (every write fails
+`MessageTooLarge`) and unable to read most inbound frames — failing confusingly mid-session rather
+than at the handshake, contradicting the class's own declared intent (:14-18, "a nonsensical
+negotiated value is rejected at the handshake"). The gateway's own validator floors its config at
+1024 (`src/ZB.MOM.WW.MxGateway.Server/Configuration/GatewayOptionsValidator.cs:10,230-233`).
+
+**Impact.** A misconfigured or buggy peer produces a session that handshakes cleanly and then
+fails every subsequent operation with per-frame size errors — the worst diagnostic shape for an
+operator, and exactly what the top-end check was added to prevent.
+
+**Design.** Mirror the gateway's validation floor: reject negotiated values below 1024 with the
+same `InvalidConfiguration` error the ceiling uses, so the failure is a handshake-time fault frame
+(`ValidateGatewayHello` → `CompleteStartupHandshakeAsync`'s `WorkerFrameProtocolException` catch →
+`TryWriteFaultAsync` + throw) instead of a mid-session mystery. Floor value: **1024**, matching
+`GatewayOptionsValidator.MinimumMaxMessageBytes` — the worker must never reject a value the
+gateway's own validator accepts as legal configuration. Rejected alternative: a 64 KiB floor
+(review's other suggestion) — safer-sounding but would turn a gateway legitimately configured
+anywhere in [1024, 65535] into a handshake failure; symmetry of the two processes' validation
+bounds is the defensible contract, and 1024 already guarantees hellos, heartbeats, acks, and
+faults fit.
+
+**Implementation.**
+1. `Ipc/WorkerFrameProtocolOptions.cs`: add
+   `public const int MinNegotiableFrameBytes = 1024;` (XML doc: matches the gateway's
+   `MinimumMaxMessageBytes` validation floor); in `AdoptNegotiatedMaxMessageBytes`, after the
+   `== 0` early return, throw `WorkerFrameProtocolException(InvalidConfiguration)` when
+   `negotiatedMaxFrameBytes < MinNegotiableFrameBytes`, message symmetrical with the ceiling text.
+2. Docs same commit: `docs/WorkerFrameProtocol.md` negotiated-max paragraph (:25-30) — state the
+   accepted range [1024, 256 MiB], 0 = keep default, out-of-range rejected at the handshake.
+
+**Verification.** On windev:
+`dotnet test src/ZB.MOM.WW.MxGateway.Worker.Tests/ZB.MOM.WW.MxGateway.Worker.Tests.csproj -p:Platform=x86 --filter "FullyQualifiedName~WorkerFrameProtocolTests|FullyQualifiedName~WorkerPipeSessionTests"`.
+New tests:
+- `WorkerFrameProtocolTests.AdoptNegotiatedMaxMessageBytes_BelowFloor_ThrowsInvalidConfiguration`
+  (512 → throws; 1024 → adopted; boundary pair).
+- `WorkerPipeSessionTests.Handshake_GatewayHelloWithTinyMaxFrameBytes_FaultsAtHandshake` — mirror
+  of the existing >256 MiB handshake test: `GatewayHello.MaxFrameBytes = 512` → fault frame
+  written, handshake throws, no message loop entered.
+
+---
+
+## WRK-25 — WRK-12 flush coalescing never engages on the event hot path   `Low` · `P2`
+
+**Finding.** The event drain loop awaits each event's `WriteAsync` — which completes only after
+that frame is written *and flushed* — before writing the next
+(`Ipc/WorkerPipeSession.cs:374-382`), so the writer's event queue holds at most one frame from the
+drain loop and every event still costs one flush. The batch flush path
+(`Ipc/WorkerFrameWriter.cs:120-124, 160-182`) only coalesces when independent producers happen to
+queue behind a blocked in-progress write. Residual of prior Done item WRK-12: the tracking-log
+claim "a burst of N events costs 1 flush not N" does not describe production behavior.
+
+**Impact.** Performance only: under event bursts (the gateway's core load) each event pays a
+semaphore round-trip and a pipe flush; the shipped coalescing mechanism sits idle on the exact
+path it was built for.
+
+**Design.** Give the drain loop a real batch entry point: `WriteBatchAsync` on the writer —
+enqueue every frame of the drained batch under one `_gate` acquisition (preserving intra-batch
+order), take the write lock once, drain (existing logic: control-first, one flush per batch),
+then await all completions. The existing single-drainer machinery then does exactly what WRK-12
+intended: N events, one flush. Guarantees preserved: control-before-event holds because the batch
+still lands in `_eventFrames` and any concurrently queued control frame is drained first by
+`DequeueNext`; intra-batch event order holds because the enqueue is atomic under `_gate` and the
+event queue is FIFO; the "written and flushed before completion" contract is unchanged. A
+per-frame rejection inside the batch (an oversized single event) surfaces from the awaited
+completions and terminates the session exactly as today — that policy is IPC-30's, deliberately
+not changed here. Rejected alternative (review's other option): fire N `WriteAsync` calls and
+`Task.WhenAll` — works (each call enqueues synchronously before its first await) but costs N
+semaphore acquisitions and N no-op drains for zero benefit over the explicit batch. Depends on
+WRK-22 landing first or together: both touch the enqueue/`DequeueNext` seam, and batch cancellation
+reuses WRK-22's tombstoning.
+
+**Implementation.**
+1. `Ipc/WorkerFrameWriter.cs`: add
+   `public async Task WriteBatchAsync(IReadOnlyList<WorkerEnvelope> envelopes, WorkerFrameWritePriority priority, CancellationToken cancellationToken = default)`
+   — build a `PendingFrame[]`, enqueue all under one `lock (_gate)`, single
+   `await _writeLock.WaitAsync(cancellationToken)` (on `OperationCanceledException`: tombstone all
+   unclaimed frames per WRK-22, rethrow), `DrainQueuedFramesAsync()`, release, then await each
+   completion in order (plain loop over the array — no LINQ needed, net48-safe either way).
+2. `Ipc/WorkerPipeSession.cs`, `RunEventDrainLoopAsync` (:374-382): replace the per-event awaited
+   loop with building the envelope list for the drained batch and one
+   `WriteBatchAsync(envelopes, WorkerFrameWritePriority.Event, cancellationToken)`. Keep the
+   priority comment, updated.
+3. Docs same commit: `docs/WorkerFrameProtocol.md` scheduling section (WRK-26, which lands earlier
+   as P1) — amend its flush-coalescing sentence to say the event drain loop submits drained
+   batches through the batch entry point, so a burst of N events costs one flush.
+
+**Verification.** On windev:
+`dotnet test src/ZB.MOM.WW.MxGateway.Worker.Tests/ZB.MOM.WW.MxGateway.Worker.Tests.csproj -p:Platform=x86 --filter "FullyQualifiedName~WorkerFrameProtocolTests|FullyQualifiedName~WorkerPipeSessionTests"`.
+New tests:
+- `WorkerFrameProtocolTests.WriteBatchAsync_FlushesOnceAndPreservesOrder` — flush-counting fake
+  stream; N event envelopes → exactly one flush, wire order = batch order, sequences monotonic.
+- `WorkerFrameProtocolTests.WriteBatchAsync_ControlFrameQueuedDuringBatch_JumpsRemainingEvents` —
+  control-before-event still holds mid-batch.
+- `WorkerPipeSessionTests.EventBurst_DrainLoopCoalescesFlushes` — fake runtime yields a 128-event
+  batch; assert the capturing stream saw one flush for the batch (this is the assertion the WRK-12
+  tracking claim needs to actually be true).
+
+---
+
+## WRK-26 — Write-priority and overflow doc drift from the WRK-07 change   `Low` · `P1`
+
+**Finding.** `docs/MxAccessWorkerInstanceDesign.md:607-613` still specifies the five-level write
+priority (faults > replies > shutdown acks > heartbeats > events) while the implementation is a
+two-class scheduler (Control FIFO, then Event — `Ipc/WorkerFrameWritePriority.cs:10-17`,
+`Ipc/WorkerFrameWriter.cs:200-216`); within Control a fault does not jump queued
+heartbeats/replies. `docs/WorkerFrameProtocol.md` describes the negotiated max (:25-30) but says
+nothing about priority classes, flush coalescing, write-time sequence stamping, or the per-frame
+vs stream-failure semantics. The adjacent overflow-policy text
+(`docs/MxAccessWorkerInstanceDesign.md:615-624`, "stop accepting new commands") is also stale
+against the implemented fail-fast (WRK-18's doc half). CLAUDE.md requires affected docs to change
+in the same commit as the source; this is the residual of Done item WRK-07. Roadmap P1 doc-drift
+batch (item 8); **discharges IPC-29**, which asks for the same `WorkerFrameProtocol.md` section.
+
+**Impact.** The component design doc actively misleads: a maintainer adding a new frame kind will
+look for a five-level queue that does not exist, and the accepted FIFO-within-control decision is
+recorded nowhere, so it reads as a defect rather than a choice.
+
+**Design.** Doc-only; make the docs describe the implemented scheduler and record the decision.
+No code change — the two-class scheduler is correct and accepted (the review's deep-dive found it
+deadlock-free and lost-wakeup-free; within-Control FIFO is bounded by the small control queue
+depth). Ordering note: this is P1 and will likely land before WRK-23/WRK-25 (P2/untierred), so the
+new text must describe HEAD truthfully at commit time — sequence numbers *are* consumed by rejected
+frames until WRK-23 lands, and event flushes are *not* yet coalesced on the drain path until
+WRK-25 lands; write those two facts as they are, and WRK-23/WRK-25 update the sentences in their
+own commits (same-commit docs rule applies to them too).
+
+**Implementation.**
+1. `docs/MxAccessWorkerInstanceDesign.md:602-624`: rewrite the "Outbound Queues" section —
+   two-class cooperative scheduler (`WorkerFrameWriter` + `WorkerFrameWritePriority`): Control
+   (faults, command replies, shutdown acks, heartbeats, hello/ready) FIFO among themselves and
+   always ahead of Event frames; enqueue-then-contend, lock-holder drains all queued frames;
+   record the accepted decision that the original five-level order was collapsed and why (control
+   queue is shallow, so intra-control FIFO delay is bounded and the machinery stays simple).
+   Rewrite the overflow paragraph to the implemented fail-fast (fault written, drain loop throws,
+   process exits — currently the generic exit code; WRK-18 still tracks the dedicated code).
+2. `docs/WorkerFrameProtocol.md`: add a "Write scheduling and sequencing" section — priority
+   classes; `Sequence` stamped at the moment of writing under the write lock so wire order and
+   stamped sequence always agree; per-frame rejections (`InvalidEnvelope`, `MessageTooLarge`,
+   version/session mismatch) fail only that frame while stream failures fail the batch and all
+   queued frames; flush coalescing across a drained batch and the "written and flushed" completion
+   contract; the rejection/sequence-gap behavior as it stands at commit time (see ordering note);
+   cancellation semantics once WRK-22 lands (or carried by WRK-22's commit if it lands later).
+3. Cross-check `gateway.md:328-330` (sequence prose) still reads true; touch only if wrong.
+
+**Verification.** Doc-only — no build impact. Sanity on any host:
+`grep -n "faults" docs/MxAccessWorkerInstanceDesign.md` shows no remaining five-level list;
+`grep -n "scheduling" docs/WorkerFrameProtocol.md` finds the new section. Follow
+`docs/style-guides/StyleGuide.md` (present tense, explain why). Confirm the IPC remediation
+register marks IPC-29 as discharged by this item.
+
+---
+
+## WRK-27 — Alarm poll bypasses the watchdog's in-flight suppression (15 s vs 75 s)   `Low` · `—`
+
+**Finding.** `MxAccess/MxAccessStaSession.cs:245-253` invokes `handler.PollOnce()` via
+`staRuntime.InvokeAsync` directly, bypassing `StaCommandDispatcher`; `CaptureHeartbeat` takes
+`CurrentCommandCorrelationId` only from the dispatcher (:364-381), so during a long `PollOnce` the
+heartbeat shows no in-flight command and the watchdog fires `StaHung` as soon as staleness exceeds
+`HeartbeatGrace` (15 s default) rather than the 75 s `HeartbeatStuckCeiling` granted to dispatched
+commands (`Ipc/WorkerPipeSession.cs:843-861`).
+
+**Impact.** A healthy-but-slow `GetXmlCurrentAlarms2` (large alarm set, busy provider) blocking
+the STA for >15 s faults a healthy session. Partially defensible — a blocked STA cannot deliver
+data events either — but the 15 s vs 75 s asymmetry between polls and commands is unintentional.
+
+**Design.** Surface poll-in-progress to the watchdog through the heartbeat snapshot as a boolean,
+and have the suppression branch honor it alongside `CurrentCommandCorrelationId`. The poll then
+gets exactly the command treatment: suppressed up to the ceiling, faulted past it (a poll that
+blocks the STA >75 s without pumping *should* fault — that is the ceiling's contract). All types
+touched are worker-internal; the heartbeat proto is unchanged, so no contract/client work.
+Rejected alternatives: (a) routing the poll through `StaCommandDispatcher` — inflates
+`PendingCommandCount`, subjects polls to the 128-entry command backpressure and
+shutdown-rejection semantics, and perturbs dispatch ordering for real gateway commands; the poll
+is infrastructure, not a command. (b) A synthetic correlation id (e.g. `"alarm-poll"`) in the
+heartbeat — visible to the gateway, which treats correlation ids as real command identifiers
+(dashboards, per-command timeout logic); misrepresenting a nonexistent command crosses an
+observability contract for a purely local suppression need.
+
+**Implementation.**
+1. `MxAccess/MxAccessStaSession.cs`: add `private volatile bool staAlarmPollInProgress;`; in the
+   delegate passed to `staRuntime.InvokeAsync` in `RunAlarmPollLoopAsync` (:247-253), set it
+   `true` immediately before `EnsureOnAlarmConsumerThread()`/`handler.PollOnce()` and clear it in
+   a `finally` — the flag flips on the STA thread exactly around the COM call.
+2. `MxAccess/WorkerRuntimeHeartbeatSnapshot.cs`: add a `bool StaCallInProgress` get-only property
+   and constructor parameter (plain ctor assignment — net48-safe; update all construction sites,
+   including tests). Named generically ("an STA call outside the dispatcher is executing") so any
+   future non-dispatcher STA work reuses it.
+3. `MxAccess/MxAccessStaSession.CaptureHeartbeat` (:364-381): pass `staAlarmPollInProgress`.
+4. `Ipc/WorkerPipeSession.ReportWatchdogFaultIfNeededAsync` (:850-861): change the suppression
+   condition to
+   `(!string.IsNullOrEmpty(snapshot.CurrentCommandCorrelationId) || snapshot.StaCallInProgress) && staleFor <= _sessionOptions.HeartbeatStuckCeiling`;
+   extend the branch comment.
+5. Docs same commit: `docs/MxAccessWorkerInstanceDesign.md` watchdog section (~:656-702) — one
+   paragraph: the alarm poll runs outside the dispatcher, advertises itself via the snapshot's
+   STA-call-in-progress flag, and receives the same grace-to-ceiling suppression as dispatched
+   commands.
+
+**Verification.** On windev:
+`dotnet build src/ZB.MOM.WW.MxGateway.Worker/ZB.MOM.WW.MxGateway.Worker.csproj -p:Platform=x86`
+then
+`dotnet test src/ZB.MOM.WW.MxGateway.Worker.Tests/ZB.MOM.WW.MxGateway.Worker.Tests.csproj -p:Platform=x86 --filter "FullyQualifiedName~WorkerPipeSessionTests|FullyQualifiedName~MxAccessStaSessionTests"`.
+New tests:
+- `WorkerPipeSessionTests.Watchdog_StaCallInProgress_SuppressedUntilCeiling` — fake runtime
+  returns a snapshot with activity stale beyond `HeartbeatGrace` but within the ceiling, empty
+  correlation id, `StaCallInProgress = true`; assert no `StaHung` fault; then a snapshot stale
+  beyond the ceiling; assert the fault fires (parity with the existing command-suppression tests).
+- `MxAccessStaSessionTests.CaptureHeartbeat_DuringAlarmPoll_ReportsStaCallInProgress` — block
+  `PollOnce` on a gate in a stub alarm handler, capture a heartbeat mid-poll, assert the flag;
+  release, assert it clears.
+
+---
+
+## WRK-28 — 10,000 drain cap is a duplicated magic constant with a comment-only sync contract   `Low` · `—`
+
+**Finding.** `Ipc/WorkerPipeSession.cs:26` (`MaxDrainEventsPerReply = 10_000`, "Kept in step with
+that public ceiling") vs `src/ZB.MOM.WW.MxGateway.Server/Grpc/MxAccessGrpcRequestValidator.cs:13`
+(`MaxDrainEventsPerRequest = 10_000`). Both projects already reference the Contracts assembly
+(`net10.0;net48`), which could own the constant.
+
+**Impact.** Conventions/maintenance: nothing enforces the sync; the next person to tune one side
+silently desynchronizes the gateway's loud rejection threshold from the worker's clamp.
+
+**Design.** Move the ceiling into the Contracts project next to the existing protocol constants in
+`GatewayContractInfo` (`src/ZB.MOM.WW.MxGateway.Contracts/GatewayContractInfo.cs`) and reference
+it from both sides. A C# `const` only — **no `.proto` change**, so no generated-code regeneration
+and no language-client impact (the cap is documented behavior, not wire schema). Land in the same
+commit cluster as WRK-21, which rewrites the exact worker lines that consume it. Rejected
+alternative: leaving two constants with a unit test asserting equality — works, but the shared
+constant removes the failure mode instead of detecting it.
+
+**Implementation.**
+1. `src/ZB.MOM.WW.MxGateway.Contracts/GatewayContractInfo.cs`: add
+   `public const uint MaxDrainEventsPerCommand = 10_000;` with XML doc naming both consumers (the
+   gateway's request-validation ceiling and the worker's per-reply clamp) and the byte-cap caveat
+   from WRK-21 (count is necessary, not sufficient).
+2. `src/ZB.MOM.WW.MxGateway.Server/Grpc/MxAccessGrpcRequestValidator.cs:13,81-85`: delete the
+   private const; reference `GatewayContractInfo.MaxDrainEventsPerCommand` (error text unchanged).
+3. `src/ZB.MOM.WW.MxGateway.Worker/Ipc/WorkerPipeSession.cs:26,545-557`: delete
+   `MaxDrainEventsPerReply`; reference the shared constant; trim the "kept in step" comment to
+   point at the shared home.
+4. Docs same commit: none required (no behavior change); if `docs/GatewayConfiguration.md` or the
+   DrainEvents prose names the literal 10,000, point it at the shared constant's home.
+
+**Verification.** Both sides build: on macOS
+`dotnet build src/ZB.MOM.WW.MxGateway.NonWindows.slnx` and
+`dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter "FullyQualifiedName~MxAccessGrpcRequestValidator"`;
+on windev
+`dotnet build src/ZB.MOM.WW.MxGateway.Worker/ZB.MOM.WW.MxGateway.Worker.csproj -p:Platform=x86`
+and
+`dotnet test src/ZB.MOM.WW.MxGateway.Worker.Tests/ZB.MOM.WW.MxGateway.Worker.Tests.csproj -p:Platform=x86 --filter "FullyQualifiedName~WorkerPipeSessionTests"`.
+Existing drain-bound tests on both sides are the regression guard; no new tests needed beyond
+WRK-21's.
+
+---
+
+## Cross-references and sequencing
+
+- **WRK-21 = IPC-23** (single defect, single fix — owned here; the IPC register should link to
+  this entry). Its `ProcessCommandAsync` backstop narrows command replies only; the oversized
+  *event-stream* frame policy remains **IPC-30** (out of scope here — a delivery-expectations
+  decision, not a mechanics fix).
+- **WRK-22 = IPC-26** (single defect, single fix — owned here). WRK-25 builds on WRK-22's
+  claim/tombstone machinery — land WRK-22 first or together.
+- **WRK-23** interacts with WRK-21: byte capping makes writer-level `MessageTooLarge` a rare
+  backstop, and WRK-23 then makes the gap-free sequence guarantee (already asserted by an existing
+  test) actually hold across that backstop.
+- **WRK-26** (P1) lands before the P2/untierred code items — its text must describe HEAD at commit
+  time; WRK-23 and WRK-25 each update the affected sentences in their own commits per the
+  same-commit docs rule. WRK-26 also discharges **IPC-29**.
+- Suggested commit clusters: (1) WRK-21 + WRK-28 [P0]; (2) WRK-26 [P1 doc batch, with TST-27 /
+  CLI-42 per roadmap item 8]; (3) WRK-22 + WRK-25 [writer seam]; (4) WRK-23; (5) WRK-24; (6)
+  WRK-27. Run the full worker suite on windev once per cluster batch, not per task
+  (`dotnet test src/ZB.MOM.WW.MxGateway.Worker.Tests/ZB.MOM.WW.MxGateway.Worker.Tests.csproj -p:Platform=x86`).
+- Prior-cycle open items touching the same code (WRK-02/03/05/08/09/10/13/14/16/17/18/19 —
+  confirmed still open by this review) remain tracked in `archreview/remediation/20-worker.md`;
+  WRK-26 deliberately folds in WRK-18's stale overflow *prose* but leaves its exit-code half open.

archreview/2026-07-12/remediation/30-contracts-ipc.md

@@ -0,0 +1,253 @@
+# Contracts & IPC Protocol — Remediation Design & Implementation (2026-07-12 cycle)
+
+Source review: [../30-contracts-ipc.md](../30-contracts-ipc.md) (main @ `4f5371f`) · Prior cycle's plan: [../../remediation/30-contracts-ipc.md](../../remediation/30-contracts-ipc.md) · Roadmap: [../00-overall.md](../00-overall.md) · Generated: 2026-07-13
+
+This cycle's findings cluster in two seams left by the prior remediation: **residual byte-size failure modes one layer out from the shipped fixes** (IPC-23/IPC-30 — the DrainEvents count cap and the per-frame-rejection machinery both stop short of the invariant "no single oversized payload may kill a session silently"), and **holes in the codegen freshness net** (IPC-24/IPC-25 — the CI Java churn-revert masks exactly the files where message-level drift lands, and the Go/Python worker bindings are demonstrably stale at HEAD with no guard). The rest is documentation drift and test-coverage blind spots.
+
+Two findings in this domain share a defect with the worker domain and split ownership the same way the review does: **IPC-23 ≡ WRK-21** and **IPC-26 ≡ WRK-22**. For those, this plan defines the *contract-level requirements and documentation* and defers the worker-code mechanics to the worker plan; the worker plan's fix must satisfy the requirements stated here. IPC-30 (oversized *event* frame) is designed and implemented fully in this plan because the decision is a protocol-policy call, but its code lands in the same worker file and the same P0 batch as WRK-21, sharing one windev verification run.
+
+All `path:line` citations were re-verified against the working tree at `4f5371f`.
+
+## Finding register
+
+| ID | Sev | Tier | Eff | Dep | Status | Title |
+|----|-----|------|-----|-----|--------|-------|
+| IPC-23 | Medium | P0 | S¹ | WRK-21 | Not started | DrainEvents bound is count-based only; byte-heavy queue still builds a session-killing reply frame (contract requirements here; fix mechanics in WRK-21) |
+| IPC-24 | Medium | P0 | S | — | Not started | CI's unconditional Java churn-revert masks real generated-code drift for message-level proto changes |
+| IPC-25 | Medium | P0 | M | — | Not started | Committed Go/Python worker bindings are stale at HEAD; no guard covers them |
+| IPC-26 | Low | P2 | S¹ | WRK-22 | Not started | Cancelled write leaves a ghost frame that is still written (contract requirement here; fix mechanics in WRK-22) |
+| IPC-27 | Low | P2 | S | — | Not started | Descriptor freshness test blind to enums, enum values, services/methods, and the Galaxy contract |
+| IPC-28 | Low | — | S | — | Not started | `docs/Grpc.md` omits the `CommandTooLarge` → `ResourceExhausted` mapping |
+| IPC-29 | Low | — | S | — | Not started | Worker writer priority scheduling and write-time sequence stamping undocumented in the frame-protocol doc |
+| IPC-30 | Low | P0 | M | WRK-21 (same file/batch) | Not started | Oversized worker→gateway event frame is session-fatal — make the death deliberate, structured, and diagnosable |
+| IPC-31 | Info | — | — | — | N/A | Gateway stamps sequence at creation, worker at write — accepted divergence; sequence is documented diagnostic-only (`gateway.md:328-330`); revisit only if sequence ever becomes load-bearing |
+| IPC-32 | Info | — | S | IPC-25 | Not started | `check-codegen.ps1` check labels miscounted (folded into the IPC-25 script edit) |
+
+¹ Effort for the work owned by *this* plan (proto comments + docs + acceptance criteria). The code mechanics are M and are tracked under WRK-21 / WRK-22 in the worker plan.
+
+---
+
+## IPC-23 — DrainEvents bound is count-based only; a byte-heavy queue still builds a session-killing reply frame   `Medium` · `P0` · fix mechanics in **WRK-21**
+
+**Finding.** `Worker/Ipc/WorkerPipeSession.cs:537-562` (`CreateDrainEventsReply`) clamps the drain to `MaxDrainEventsPerReply = 10_000` (`:26`) but never sizes the reply. At the default negotiated frame max (16 MiB + 64 KiB), events averaging above ~1.7 KiB overshoot `MaxMessageBytes`; the writer correctly rejects the frame per-frame (`Worker/Ipc/WorkerFrameWriter.cs:250-255`, `IsPerFrameRejection` `:192-198`) so the stream survives, but the exception propagates from `WriteControlReplyAsync` (`:481,485-496`) through `DispatchGatewayEnvelopeAsync` (`:406`) into `RunMessageLoopAsync` (`:280`) and out of `RunAsync` — the worker exits, and because `runtimeSession.DrainEvents(maxEvents)` already removed the events from the queue (`:551`), they are lost. This is the prior IPC-04 fix narrowed, not closed.
+
+**Impact.** A diagnostics command (`DrainEvents`, `max_events = 0` or any large value) against an array-heavy queue — exactly the payload profile this gateway exists for — kills the session and destroys the drained events. Violates the invariant the roadmap states for this P0 item: *no diagnostics command may be session-fatal*.
+
+**Design (contract-level requirements — the WRK-21 implementation must satisfy all of these).**
+
+1. **R1 — Reply fits the negotiated frame.** Every worker→gateway control reply, `DrainEventsReply` included, must serialize (as a full `WorkerEnvelope`) within the negotiated `MaxMessageBytes`. Concretely: `CreateDrainEventsReply` must stop packing when the running `reply.CalculateSize()` plus the next event's serialized size plus a fixed envelope-overhead allowance would exceed the cap. The byte cap binds *in addition to* the existing `MaxDrainEventsPerReply` count cap, whichever is hit first.
+2. **R2 — No event loss on truncation.** Events that do not fit the current reply must remain in (or be returned to) the event queue in order — the drain must be incremental (peek-pack-commit or drain-one-at-a-time), not bulk-drain-then-pack. Losing undelivered events would silently break the event stream's faithfulness.
+3. **R3 — Truncation is expressible with the existing contract; no new field.** `DrainEventsReply` (`mxaccess_gateway.proto:678-680`) already permits returning fewer events than requested or available; the caller contract is *drain iteratively until an empty reply*. An additive `uint32 remaining`/`bool truncated` field was considered and **rejected for this pass**: it forces a full five-client regeneration fan-out (plus descriptor + `Generated/` + Rust vendored copies) for a diagnostics nicety the loop semantics already provide. Revisit only if operators need one-shot depth reporting — `WorkerHeartbeat.outbound_event_queue_depth` (`mxaccess_worker.proto:99`) already reports depth out-of-band.
+4. **R4 — Byte-cap semantics are documented at the contract.** Proto comments and `docs/WorkerFrameProtocol.md` must state the rule (see Implementation).
+
+**Implementation.** (Ordered; steps 1–2 are this plan's deliverable and should land in the *same commit* as the WRK-21 code fix so the proto-comment regeneration wave happens exactly once.)
+
+1. `src/ZB.MOM.WW.MxGateway.Contracts/Protos/mxaccess_worker.proto`: extend the `GatewayHello.max_frame_bytes` comment (`:45-50`) with one sentence: *every worker→gateway frame — events, heartbeats, faults, and control replies including DrainEvents — must serialize within this limit; reply builders truncate to fit rather than emit an oversized frame.*
+2. `src/ZB.MOM.WW.MxGateway.Contracts/Protos/mxaccess_gateway.proto` (`DrainEventsReply`, `:678-680`): add a comment: *the reply is bounded by both a server-side count cap and the negotiated worker-frame byte cap; a reply may therefore carry fewer events than `max_events` and fewer than are queued. Callers drain iteratively until an empty reply.*
+3. Comment-only proto edits still trigger the full regen chain (comments surface in generated C# XML docs, Go doc comments, Java javadoc, and the Rust vendored copies are byte-compared by `check-codegen.ps1` Check 3):
+   - regenerate `Contracts/Generated/` (`dotnet build src/ZB.MOM.WW.MxGateway.Contracts/ZB.MOM.WW.MxGateway.Contracts.csproj`, after `rm src/ZB.MOM.WW.MxGateway.Contracts/Generated/*.cs`) and **commit** (net48 CS0246 rule);
+   - refresh `clients/rust/protos/*.proto` (byte-copy from Contracts/Protos);
+   - regenerate Go (`pwsh clients/go/generate-proto.ps1`, pinned tools per IPC-25), Java (`gradle generateProto` on the pinned toolchain), Python (`pwsh clients/python/generate-proto.ps1` — comment-only edits normally produce *no* Python delta since `_pb2` embeds a source-info-free descriptor; commit only a real delta, revert noise);
+   - regenerate the descriptor set (`pwsh scripts/publish-client-proto-inputs.ps1` with pinned protoc 34.1) so grpcurl users see the new comments.
+4. `docs/WorkerFrameProtocol.md`: add the byte-cap rule to the size-limits paragraph (`:19-30`): control replies are size-aware-packed; `DrainEvents` truncates to fit; drain-until-empty caller contract. (This lands in the same section IPC-29 adds — do the two doc edits together.)
+5. `docs/Grpc.md` DrainEvents rules row (`:180` area): note the reply may be truncated by byte size as well as count; drain iteratively.
+6. Worker code fix + tests: **deferred to WRK-21** (size-aware packing loop in `CreateDrainEventsReply`, incremental drain to satisfy R2, worker test that a queue of ~10,000 large events drains over multiple replies with the session alive and no event lost).
+
+**Verification.**
+- Contract side (this plan): `pwsh scripts/check-codegen.ps1` passes after the regen wave (all checks); `dotnet build src/ZB.MOM.WW.MxGateway.NonWindows.slnx`; `dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter FullyQualifiedName~ClientProtoInputTests`.
+- Mechanics (WRK-21, on windev via the isolated `C:\build` worktree): `dotnet build src/ZB.MOM.WW.MxGateway.Worker/ZB.MOM.WW.MxGateway.Worker.csproj -p:Platform=x86` and `dotnet test src/ZB.MOM.WW.MxGateway.Worker.Tests/ZB.MOM.WW.MxGateway.Worker.Tests.csproj -p:Platform=x86 --filter FullyQualifiedName~WorkerPipeSession` — including the WRK-21 repro test (enqueue ~10,000 large events, `DrainEvents max_events=0`, assert session survives and total drained count equals enqueued count across iterative replies).
+
+---
+
+## IPC-24 — CI's unconditional churn-revert step masks real Java generated-code drift for message-level proto changes   `Medium` · `P0`
+
+**Finding.** `.gitea/workflows/ci.yml:114-119` runs `git checkout -- clients/java/.../MxaccessGateway.java` and `.../MxaccessWorker.java` unconditionally before the "Verify generated tree is clean" step (`:120-121`). The Java bindings are single-file aggregates (`java_multiple_files` unset), so any message/field-level proto change lands **only** in those two files — a `.proto` edit committed without regenerating the Java client passes CI because the freshly regenerated (correct) files are reverted to the stale committed ones before the diff check. Only service-level changes (which touch the separate `*Grpc.java` stubs) would still be caught. The step's own comment scopes it to "when no `.proto` changed", but nothing enforces that condition.
+
+**Impact.** The `checkGeneratedClean` intent from IPC-09 is neutered for the most common contract-change class. Combined with IPC-25, message-level drift currently has **zero** automated coverage in three of five clients.
+
+**Design.** The churn the revert step works around is protobuf-gencode-version drift (`validateProtobufGencodeVersion(... major/minor/patch ...)` stanzas rewritten when the regenerating toolchain differs from the one that produced the committed files — repo memory `project_java_generated_churn`, ~64k lines). But the toolchain is now fully pinned: `clients/java/build.gradle:8,11` pin `grpcVersion = 1.76.0` / `protobufVersion = 4.33.1`, the protoc artifact and grpc plugin derive from those pins (`zb-mom-ww-mxgateway-client/build.gradle:40-46`), and the committed `MxaccessWorker.java` already stamps gencode 4.33.1. Under matching pins, regeneration should be **byte-identical** — meaning the revert step is a fossil from the pre-pin era, and the correct fix is to *delete it* and let `git diff --exit-code` be the real gate.
+
+Discriminator design, in preference order:
+
+1. **Preferred — prove the churn is gone under the pin and delete the revert.** Empirically regenerate on the pinned toolchain and check the tree. If clean, remove the revert step outright. Simplest, no new machinery, converts the Java job into a true drift gate.
+2. **Fallback (only if churn persists) — diff modulo the known volatile lines.** Replace the unconditional checkout with a script step: after regeneration, take `git diff -U0` over the two aggregates, strip additions/deletions matching the volatile patterns (`validateProtobufGencodeVersion`, the `/* major= */ … /* patch= */` literals, generator-version header comments); empty residue → churn → revert; non-empty residue → **fail** with "Java generated bindings are stale — regenerate with the pinned toolchain and commit". Put the script in `scripts/check-java-generated.sh` so local use matches CI.
+3. **Rejected — gate the revert on the push/PR diff containing no `*.proto` change** (`git diff --name-only $BASE..HEAD -- '*.proto'`). Two defects: the base-ref plumbing differs per trigger (push `github.event.before` vs PR base vs the nightly `schedule` run, which has no meaningful base), and it discriminates on the *delta*, not the *tree* — a stale binding merged in an earlier commit passes every subsequent run forever. The tree-level checks above are strictly stronger.
+
+**Implementation.**
+1. Prove/disprove churn locally (the Java client builds on the Mac now — memory `project_java_build_host`): `cd clients/java && JAVA_HOME=/opt/homebrew/opt/openjdk@17 gradle generateProto && git status --porcelain -- src/main/generated`. Expected: clean (pins match the committed gencode stamps).
+2. If clean: delete the "Revert spurious protobuf-version churn" step (`.gitea/workflows/ci.yml:114-119`), keep `Verify generated tree is clean` (`:120-121`) as-is. Also delete the churn-revert comment at `:92-95`.
+3. If churn persists: add `scripts/check-java-generated.sh` implementing design 2 and replace `ci.yml:114-119` with a step invoking it.
+4. Same-commit doc updates (docs-change-with-source rule): `docs/GatewayTesting.md:421-425` (CI section prose describing the revert), `docs/ClientProtoGeneration.md:96-100` (the "revert that one file when no `.proto` changed" caveat), and the `checkGeneratedClean` comment block in `clients/java/zb-mom-ww-mxgateway-client/build.gradle:61-70` (the "CI reverts that one file" caveat). Update the repo memory note `project_java_generated_churn` guidance if step 1 shows the churn is gone.
+5. Negative-path proof on a throwaway branch: add a scratch field to `mxaccess_worker.proto`, regenerate *only* `Contracts/Generated/` (not Java), push, and confirm the `java` CI job now **fails** at the diff step. Delete the branch.
+
+**Verification.**
+- Local: step 1's `git status --porcelain` output recorded in the PR description (clean or not — that decides branch 2 vs 3).
+- CI-run evidence: a green `java` job on the fix branch (churn really is gone / discriminator correctly classifies), plus the deliberately-red run from step 5 proving message-level drift is now caught.
+- No .NET/proto surface changes → no gateway/worker builds needed beyond CI's normal jobs.
+
+---
+
+## IPC-25 — Committed Go and Python worker bindings are stale at HEAD, and no guard covers them   `Medium` · `P0`
+
+**Finding.** `clients/go/internal/generated/mxaccess_worker.pb.go` (last regenerated 2026-05-23, commit `397d3c5`) and `clients/python/src/zb_mom_ww_mxgateway/generated/mxaccess_worker_pb2.py` (2026-06-18, `e328758`) both predate the 2026-07-09 `max_frame_bytes` addition — `grep -c max_frame_bytes` returns 0 for both at HEAD, while the Java aggregate has it (×6). The freshness net covers `Contracts/Generated` (check-codegen Check 2), the descriptor (Check 1 + `ClientProtoInputTests`), Rust vendored protos (Check 3), and Java (`checkGeneratedClean`) — nothing compares the committed Go/Python bindings against the protos, and their unit tests don't exercise the worker types, so CI stays green.
+
+**Impact.** Functional impact today is nil (no language client consumes `GatewayHello`), but the published packages misrepresent the wire contract, and the CLAUDE.md verification-matrix rule ("regenerate every generated client touched by the contract") was violated with zero signal — the exact silent-drift class IPC-01/IPC-09 were meant to end.
+
+**Design.** Two parts: (1) regenerate and commit both binding sets now; (2) close the guard hole with a **Check 4** in `scripts/check-codegen.ps1` that regenerates Go and Python bindings via the existing per-client scripts and fails on any git diff — the same regenerate-and-diff pattern Check 2 uses for `Generated/`. The per-client scripts already carry the churn-tolerance machinery this needs: `clients/python/generate-proto.ps1:36-48` hard-asserts grpcio-tools 1.80.0 (so a regen is deterministic and a diff means real drift, not toolchain noise), and `clients/go/generate-proto.ps1:9-43` resolves pinned-on-PATH tools. The one gap: the Go script does not assert plugin versions, and plugin-version drift stamps header churn (`protoc-gen-go v1.36.11` / `protoc-gen-go-grpc v1.6.2` in the committed headers) — add the same fail-fast version assertion the Python script has, so Check 4 cannot false-fail (or worse, mask drift under churn) on an off-pin machine.
+
+Rejected alternatives: a semantic grep for known-new symbols (catches this instance, not the class); folding Go/Python regen into `ClientProtoInputTests` (the test project must stay toolchain-free — that is its documented value, `docs/ClientProtoGeneration.md:76-81`); comparing committed bindings against the descriptor set semantically (a weaker parse-level check that misses generator-behavior drift and requires per-language descriptor parsing for no gain over regenerate-and-diff).
+
+**Implementation.**
+1. **Pin the Go generators in the script.** `clients/go/generate-proto.ps1`: after `Resolve-Tool`, assert `& $protocGenGo --version` equals `protoc-gen-go v1.36.11` and `& $protocGenGoGrpc --version` equals `protoc-gen-go-grpc 1.6.2` (match the committed header stamps); throw with an install hint (`go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11`, `go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.6.2`) on mismatch. protoc itself: warn-if-off-pin (34.1) matching `publish-client-proto-inputs.ps1` semantics.
+2. **Regenerate Go** (macOS or Windows; PATH-first scripts work on both):
+   ```
+   go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11
+   go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.6.2
+   pwsh clients/go/generate-proto.ps1
+   ```
+   Expected delta: `max_frame_bytes` additions confined to `mxaccess_worker.pb.go` (descriptor bytes + `MaxFrameBytes` accessor). If other files churn, the pins are wrong — stop and reconcile before committing. Then from `clients/go`: `gofmt -l .` (must be empty), `go build ./...`, `go test ./...`.
+3. **Regenerate Python** with the pinned toolchain (memory `project_python_client_regen_pin`):
+   ```
+   python -m pip install 'grpcio-tools==1.80.0'   # bundles protobuf 6.31.1 codegen
+   pwsh clients/python/generate-proto.ps1          # Assert-GrpcioToolsVersion enforces the pin
+   ```
+   Expected delta: the serialized-descriptor line(s) in `mxaccess_worker_pb2.py` only — keep only the real descriptor delta; revert any file the regen did not meaningfully change. Then from `clients/python`: `python -m pytest`.
+4. **Java untouched** (no `.proto` changed): if gradle ran incidentally, `git checkout -- clients/java/src/main/generated` (memory `project_java_generated_churn`; goes away if IPC-24 step 1 confirms the pin killed the churn).
+5. **Add Check 4 to `scripts/check-codegen.ps1`:** "Go and Python client bindings match a fresh regeneration" — invoke `clients/go/generate-proto.ps1` and `clients/python/generate-proto.ps1`, then fail on non-empty `git status --porcelain -- clients/go/internal/generated clients/python/src/zb_mom_ww_mxgateway/generated`. Tool-missing must **fail** the check, not skip it (a skipped guard is this finding). Update the script's header comment ("Three checks" → four, list the new check) and **relabel all check banners `1/4`…`4/4`** (absorbs IPC-32).
+6. **CI:** in `.gitea/workflows/ci.yml` `portable` job, before the `Codegen / descriptor freshness` step (`:60-62`), add the toolchain installs Check 4 needs: `go install .../protoc-gen-go@v1.36.11`, `go install .../protoc-gen-go-grpc@v1.6.2` (Go is already set up at `:28-30`), and `python -m pip install 'grpcio-tools==1.80.0'` (Python at `:32-34`; protoc 34.1 already installed at `:43-47`).
+7. **Same-commit docs:** `docs/ClientProtoGeneration.md` pinned-versions table (`:88-95`): add `protoc-gen-go v1.36.11` / `protoc-gen-go-grpc v1.6.2` rows and change the Go/Python guard column to "check-codegen Check 4"; `docs/Contracts.md` cross-link if it enumerates the checks.
+8. **Publishing note (cross-domain, CLI):** the currently *published* Go/Python packages carry the stale worker bindings. Do not republish from this fix alone — CLI-39 (version constants aligned to an already-published 0.1.2) must be resolved first; flag to the clients-domain plan.
+
+**Verification.**
+- `pwsh scripts/check-codegen.ps1` locally: all four checks green post-commit; then a negative run — touch a scratch field in `mxaccess_worker.proto`, regenerate only `Contracts/Generated/`, rerun and confirm **Check 4 fails** naming both stale directories; revert.
+- `grep -c max_frame_bytes clients/go/internal/generated/mxaccess_worker.pb.go clients/python/src/zb_mom_ww_mxgateway/generated/mxaccess_worker_pb2.py` → both non-zero.
+- `cd clients/go && gofmt -l . && go build ./... && go test ./...`; `cd clients/python && python -m pytest`.
+- CI-run evidence: green `portable` job on the fix branch with the Check 4 banner visible in the log.
+- Nothing here needs windev (no worker/x86 surface).
+
+---
+
+## IPC-26 — Worker frame writer: cancelling while waiting for the write lock leaves a ghost frame that is still written   `Low` · `P2` · fix mechanics in **WRK-22**
+
+**Finding.** `Worker/Ipc/WorkerFrameWriter.cs:87-113`: the `PendingFrame` is enqueued (`:88-98`) *before* `await _writeLock.WaitAsync(cancellationToken)` (`:103`). If the caller's token fires while waiting, `WriteAsync` throws `OperationCanceledException` but the frame stays queued — the next lock-holder writes it anyway (`DequeueNext` `:200-216` has no tombstone concept), its `Completion` resolves unobserved, and during shutdown leftover cancelled event frames can drain *behind* the `WorkerShutdownAck` that is nominally the last frame. If no writer ever follows, the frame lingers unwritten with an unobserved completion.
+
+**Impact.** Benign today (the gateway drops post-shutdown/unknown frames), but the "cancelled means not written" contract is silently violated, and post-ack event emission is exactly the kind of latent seam a future gateway-side strictness change would trip over.
+
+**Design (contract-level requirement — the WRK-22 implementation must satisfy this).** *A `WriteAsync` that completes with `OperationCanceledException` guarantees its frame is never subsequently written to the stream.* The recommended shape (owned by WRK-22): on cancellation, tombstone the pending frame — `Completion.TrySetCanceled()` under `_gate`, and have `DequeueNext` (or the drain loop) skip frames whose completion is already terminal. Removal-by-scan is equivalent; tombstoning avoids O(n) queue rewrites. Additionally, `WorkerShutdownAck` remains the last frame written: since the ack is a control frame and the scheduler drains control-before-event, the tombstone rule alone restores this (cancelled event frames were the only leak path). No proto or gateway change; no doc change beyond the IPC-29 section noting the cancellation semantics once WRK-22 lands.
+
+**Implementation.**
+1. Code + tests: **deferred to WRK-22** (`Worker/Ipc/WorkerFrameWriter.cs`; worker test: cancel a `WriteAsync` blocked on the lock, then drive another write, assert the cancelled envelope never reaches the stream and its task is `Canceled`).
+2. This plan: when writing the IPC-29 doc section, include one sentence — *a write cancelled while queued is tombstoned and never reaches the wire* — phrased to land in the same commit as the WRK-22 fix (do not document it before it is true).
+
+**Verification.** WRK-22's tests on windev: `dotnet test src/ZB.MOM.WW.MxGateway.Worker.Tests/ZB.MOM.WW.MxGateway.Worker.Tests.csproj -p:Platform=x86 --filter FullyQualifiedName~WorkerFrameProtocolTests`. Doc cross-check against the shipped behavior.
+
+---
+
+## IPC-27 — Descriptor freshness test checks messages and fields only; enums, enum values, services/methods, and the Galaxy contract are blind spots   `Low` · `P2`
+
+**Finding.** `src/ZB.MOM.WW.MxGateway.Tests/Contracts/ClientProtoInputTests.cs:35-52` collects only `{message}` and `{message}/{field}` symbols and enumerates only `MxaccessGatewayReflection.Descriptor` and `MxaccessWorkerReflection.Descriptor`. A new enum value (e.g. a new `MxCommandKind` case — the most likely future contract change), a new RPC, or any change confined to `galaxy_repository.proto` would not redden the test. The script-side `-Check` in CI does catch all of these (full descriptor compare), so the gate as-a-whole holds — but `docs/ClientProtoGeneration.md:76-81` presents the test as the protoc-free **primary** gate, and on any runner without protoc it is the only one.
+
+**Impact.** The documented primary guard is weaker than its documentation claims; a protoc-less environment (e.g. a future runner change) silently loses enum/service/Galaxy coverage.
+
+**Design.** Extend the same reflection walk — all surfaces are available on `FileDescriptor` without protoc:
+- **Enums:** top-level `file.EnumTypes` and nested `message.EnumTypes`, collecting `{enumFullName}` and `{enumFullName}/{valueName}` (value *presence* suffices; a renumber would be a non-additive change caught by review and the script check — asserting numbers here would duplicate that at the cost of a noisier failure mode).
+- **Services/methods:** `file.Services`, collecting `{serviceFullName}` and `{serviceFullName}/{methodName}`.
+- **Galaxy:** add `GalaxyRepositoryReflection.Descriptor` to the enumerated files (it is compiled into Contracts — `galaxy_repository.proto` is retained there as the client-codegen source per the csproj comment).
+The published-side collection (`CollectPublishedSymbols`, `:61-79`) grows matching walks over `FileDescriptorProto.EnumType`/`Service` and `DescriptorProto.EnumType`. Keep the flat string-set comparison — it stays order-insensitive and protoc-free.
+
+**Implementation.**
+1. `ClientProtoInputTests.cs`: extend `Descriptor_ContainsEveryContractMessageAndField` (rename to `Descriptor_ContainsEveryContractSymbol`) per the design; failure message keeps listing missing symbols.
+2. `docs/ClientProtoGeneration.md:76-81`: update the prose from "message or field" to the full symbol coverage — same commit.
+3. No proto, config, or CI change (the test already runs in the portable job via the gateway test step).
+
+**Verification.**
+- `dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter FullyQualifiedName~ClientProtoInputTests` (macOS).
+- Red-path proof during development: extract the pre-IPC-01 stale protoset (`git show 0f88a95:clients/proto/descriptors/mxaccessgw-client-v1.protoset > <scratch>/stale.protoset`), point the test's path at it temporarily, and confirm it fails naming (at minimum) the `max_frame_bytes` field and — with the extension — any post-`0f88a95` enum values. Restore before commit.
+
+---
+
+## IPC-28 — `docs/Grpc.md` exception-mapping prose omits `CommandTooLarge` → `ResourceExhausted`   `Low` · `—`
+
+**Finding.** `Grpc/MxAccessGatewayService.cs:955` maps `WorkerClientErrorCode.CommandTooLarge` to `ResourceExhausted` (the IPC-03 per-correlation fix), but `docs/Grpc.md:249` still enumerates only `CommandTimeout`/`GatewayShutdown`/`InvalidState`/`ProtocolViolation` with "unmapped codes fall through to `Unavailable`" — a reader concludes an oversized command surfaces as `Unavailable`. (`docs/GatewayConfiguration.md:120-129` does document the headroom rule.)
+
+**Impact.** Doc drift on a shipped, client-visible status mapping; violates the docs-change-with-source rule retroactively.
+
+**Design.** Doc-only. Add `CommandTooLarge → ResourceExhausted` to the `WorkerClientException` mapping sentence, and one line in the Invoke handler section: an accepted gRPC payload whose worker envelope would exceed the pipe frame limit fails *that command* with `ResourceExhausted` (per-correlation, session stays alive) — cross-reference `docs/GatewayConfiguration.md`'s headroom rule.
+
+**Implementation.**
+1. `docs/Grpc.md:249`: extend the mapping prose.
+2. `docs/Grpc.md` Invoke section: add the oversized-payload sentence.
+3. If IPC-23/WRK-21 lands first, fold its DrainEvents-truncation doc row (IPC-23 step 5) into the same edit pass.
+
+**Verification.** Doc-only; cross-check the prose against the `switch` in `Grpc/MxAccessGatewayService.cs:950-960` after editing.
+
+---
+
+## IPC-29 — Worker writer priority scheduling and write-time sequence stamping undocumented in the frame-protocol doc   `Low` · `—`
+
+**Finding.** The worker writer is now a cooperative priority scheduler (control frames drain ahead of event frames, `Worker/Ipc/WorkerFrameWriter.cs:11-17,116-124`) that stamps `Sequence` at the moment of writing under the lock (`:238-240`) and coalesces flushes per batch (`:125-182`). These are protocol-relevant semantics: a per-frame rejection consumes a sequence number (stamped at `:240` before the size check at `:250` throws), so the wire sequence has gaps after rejections, and event frames can be arbitrarily delayed behind a control backlog. `docs/WorkerFrameProtocol.md:1-7` still describes the frame layer as only "message boundaries, size limits, protobuf parsing, and envelope validation"; `gateway.md:328-330` covers the diagnostic-sequence half but not scheduling.
+
+**Impact.** Anyone implementing an alternate-language worker (contemplated in `gateway.md`) or debugging sequence gaps from a pipe capture has no doc for the shipped semantics — the exact drift class the repo's docs-change-with-source rule exists to prevent.
+
+**Design.** Doc-only: add a "Write scheduling and sequencing" section to `docs/WorkerFrameProtocol.md` covering (a) the two-class priority queues and the control-before-event drain guarantee; (b) enqueue-then-contend locking, single-writer drain; (c) sequence stamped at the point of writing so wire order and sequence order always agree on the worker side; (d) sequence gaps after per-frame rejections are expected and benign (sequence is diagnostic-only — cross-ref `gateway.md:328-330`; the gateway side stamps at creation, see IPC-31, so its sequence order is *not* a wire-order guarantee); (e) flush coalescing with the written-and-flushed completion contract; (f) once WRK-22 lands, the cancellation tombstone rule (see IPC-26 — do not document before it ships).
+
+**Implementation.**
+1. `docs/WorkerFrameProtocol.md`: insert the section after "Envelope Validation"; fold in IPC-23's byte-cap sentence (step 4 there) if landing together.
+2. Optionally one cross-reference line in `gateway.md:328-330` pointing to the new section. No code change.
+
+**Verification.** Doc-only; cross-check each claim against `Worker/Ipc/WorkerFrameWriter.cs` and `WorkerFrameWritePriority.cs` at time of writing.
+
+---
+
+## IPC-30 — Oversized worker→gateway event frame is still session-fatal despite the per-frame rejection machinery   `Low` · `P0` (lands with WRK-21 in the same file/batch)
+
+**Finding.** `Worker/Ipc/WorkerPipeSession.cs:374-382`: the event drain loop awaits each event write; a `MessageTooLarge` per-frame rejection from the writer (an MXAccess array event serializing above the negotiated max) throws out of `RunEventDrainLoopAsync`, is rethrown by the message loop's `await eventDrainTask` (`:294`), and terminates `RunAsync` — the whole session dies for one unsendable event, with no fault frame and no record of *which* event. The writer survives the rejection (`WorkerFrameWriter.cs:141-147`); the session dies anyway one layer up, wasting the machinery.
+
+**Impact.** One pathological tag (a huge array) kills the session for every item the client has, and the operator gets a generic protocol-violation exit with no way to find the offending tag.
+
+**Design — position taken: session-fatal is the correct contract; the defect is that the death is unstructured.** Rationale: an event above the negotiated frame max is **undeliverable end-to-end** — the negotiated pipe max sits only `EnvelopeOverheadReserveBytes` (64 KiB, `Server/Workers/WorkerFrameProtocolOptions.cs:20`) above the public gRPC cap, so any event the pipe rejects would also be rejected by the client-facing gRPC stream. Given that, the honest options reduce to how the session dies, not whether:
+- **Silently drop and continue — rejected.** The event stream would silently stop being faithful; that breaks the delivery expectation behind every subscription, and the repo's "don't synthesize events" rule bars papering over it with a fabricated placeholder.
+- **Drop + synthesized loss-marker event — rejected.** `ReplayGap` is a gateway-lifecycle stream element (replay-ring eviction), not a worker event; reusing it would lie about replay state, and a *new* loss-marker event is exactly the synthesized event the conventions prohibit.
+- **Chunked/segmented event frames — rejected.** Contract surgery plus reassembly state on both sides for a case that indicates misconfiguration (`MaxMessageBytes` too small for the workload); MXAccess parity treats an event as atomic.
+- **Chosen: deliberate, structured session-fatal.** Catch the per-frame `MessageTooLarge` rejection at the drain-loop seam, log the event identity — family, item/tag name, worker sequence, serialized size; **never the value payload** (redaction rule) — write a `WorkerFault` so the gateway sees a structured fault instead of a raw pipe drop, then exit. This mirrors the existing fail-fast precedent (`WORKER_FAULT_CATEGORY_QUEUE_OVERFLOW` faults the session on event-queue overflow) and the drain loop's existing fault pattern (`WorkerPipeSession.cs:356-365`). Operator remediation is config (`MxGateway:Worker:MaxMessageBytes`), and now the log names the tag that needs it.
+- **Fault category: reuse `WORKER_FAULT_CATEGORY_PROTOCOL_VIOLATION` (`mxaccess_worker.proto:130`)** with a diagnostic message carrying the event identity and sizes. A dedicated additive enum value (`..._EVENT_TOO_LARGE = 12`) was considered and rejected *for this pass*: it triggers the full five-client + descriptor + `Generated/` regen fan-out for a distinction the diagnostic message already conveys; revisit if dashboards need to alert on it specifically. This keeps IPC-30 proto-neutral.
+
+**Implementation.** (Same file and same P0 batch as WRK-21 — one commit or adjacent commits, one windev verification run.)
+1. `Worker/Ipc/WorkerPipeSession.cs`, `RunEventDrainLoopAsync` (`:374-382`): wrap the per-event `WriteAsync` in a `catch (WorkerFrameProtocolException ex) when (ex.ErrorCode == WorkerFrameProtocolErrorCode.MessageTooLarge)`. In the handler: log event identity + serialized size (structured log, no values); set `_state = WorkerState.Faulted`; `await TryWriteFaultAsync(new WorkerFault { Category = ProtocolViolation, CommandMethod = "EventDrain", DiagnosticMessage = "<family> event for '<item>' (worker sequence N) serialized to X bytes, exceeding the negotiated frame max Y; raise MxGateway:Worker:MaxMessageBytes for this workload" }, ct)`; then rethrow/throw `InvalidOperationException` so `RunAsync` exits exactly as today — the fault frame is small and control-priority, so it precedes the exit. Other per-frame rejection codes (`InvalidEnvelope` etc.) keep today's behavior — they indicate worker bugs, not workload size.
+2. Gateway side: no change — `WorkerClient`'s existing `WorkerFault` handling already surfaces structured faults to the session and dashboard.
+3. Worker test (`Worker.Tests`, x86): fake runtime session emitting one event whose envelope exceeds a small test `MaxMessageBytes`; assert (a) a `WorkerFault` frame with `ProtocolViolation` and the item name in `DiagnosticMessage` is written before the pipe session ends, (b) the writer stream is not corrupted (a subsequent well-formed frame parse on the captured stream), (c) the session task completes faulted.
+4. Same-commit docs: `docs/WorkerFrameProtocol.md` (in/next to the IPC-29 section): oversized-event policy — undeliverable end-to-end, session faults with a structured `WorkerFault` naming the event, remediation is `MaxMessageBytes` config. `docs/MxAccessWorkerInstanceDesign.md` event-drain description gets the same one-paragraph policy.
+
+**Verification.**
+- windev (isolated `C:\build` worktree, memory `project_windev_worker_verify`): `dotnet build src/ZB.MOM.WW.MxGateway.Worker/ZB.MOM.WW.MxGateway.Worker.csproj -p:Platform=x86`; `dotnet test src/ZB.MOM.WW.MxGateway.Worker.Tests/ZB.MOM.WW.MxGateway.Worker.Tests.csproj -p:Platform=x86 --filter FullyQualifiedName~WorkerPipeSession` (covers this and WRK-21 in one run).
+- Gateway fake-worker regression: `dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter FullyQualifiedName~WorkerClient` (fault propagation unchanged).
+- Until TST-25's automated Windows tier exists, record the windev run (pass counts, date) in the tracking log — CI cannot evidence this one.
+
+---
+
+## IPC-31 — Gateway envelope sequence stamped at creation, not at write   `Info` · N/A
+
+**Finding.** `Server/Workers/WorkerClient.cs:1039` stamps `Sequence` via `Interlocked.Increment` inside `CreateEnvelope`; concurrent `InvokeAsync` callers can enqueue out of stamp order, so gateway→worker wire order can disagree with sequence order — the opposite semantics from the worker's write-time stamping (`Worker/Ipc/WorkerFrameWriter.cs:238-240`).
+
+**Disposition: N/A — no action.** `gateway.md:328-330` already declares sequence a per-sender diagnostic aid, never validated; both validators ignore it; the divergence is harmless and now explicitly documented by the IPC-29 doc section (which states the two sides' stamping points). If sequence ever becomes load-bearing, stamp in the gateway write loop (`WorkerClient.cs:390-397`) — recorded here so the next editor finds the decision.
+
+---
+
+## IPC-32 — `check-codegen.ps1` check labels are miscounted   `Info` · `—` (rides IPC-25)
+
+**Finding.** `scripts/check-codegen.ps1:30,42,68` print "Check 1/2", "Check 2/2", then "Check 3/3" — the middle labels predate the third check. Cosmetic; CI log readers see a miscounted progression.
+
+**Impact.** Log-readability only.
+
+**Design/Implementation.** Folded into the IPC-25 script edit: when Check 4 is added, relabel every banner `1/4`…`4/4` and update the header comment's check list in the same change (IPC-25 implementation step 5). If IPC-25 were descoped (it must not be — it is P0), the standalone fix is relabeling `1/3`, `2/3`, `3/3`.
+
+**Verification.** Banner text in the `pwsh scripts/check-codegen.ps1` output during the IPC-25 verification run; visible in the CI `portable` job log.
+
+---
+
+## Cross-domain dependencies and sequencing
+
+- **WRK-21 / IPC-23 / IPC-30 cluster (P0):** one worker-side change set in `Worker/Ipc/WorkerPipeSession.cs` (+ the proto-comment/doc wave from IPC-23). Land as one batch so the x86 build, `WorkerPipeSession`-filtered test run on windev, and the proto regen fan-out each happen once. WRK-21 owns the size-aware DrainEvents packing (must satisfy IPC-23's R1–R3); IPC-30's structured-fault seam is in the same drain loop.
+- **WRK-22 / IPC-26 (P2):** worker plan owns the tombstone fix; this plan's IPC-29 doc section documents the cancellation semantics only once it ships.
+- **IPC-24 + IPC-25 vs TST-25 (CI):** both edit `.gitea/workflows/ci.yml` (java job step removal; portable job toolchain installs). TST-25's Windows-tier work will extend the same file — coordinate branches to avoid workflow-merge conflicts, and note that everything windev-verified in this plan (IPC-23/30, WRK-21/22) has no CI evidence until TST-25 lands; record windev runs in the tracking log meanwhile.
+- **IPC-25 vs CLI-39 (publishing):** regenerated Go/Python bindings must not be republished to Gitea until the clients-domain version-collision finding (CLI-39) is resolved.

archreview/2026-07-12/remediation/40-security-dashboard.md

@@ -0,0 +1,214 @@
+# Security & Dashboard — Remediation Design & Implementation (2026-07-12 cycle)
+
+Source review: [40-security-dashboard.md](../40-security-dashboard.md) · Baseline: `main` @ `4f5371f` · Generated: 2026-07-13
+
+This document turns the six NEW findings of the 2026-07-12 security/dashboard re-review (SEC-31…SEC-36) into buildable remediation entries, in the same per-finding format as the prior cycle's [archreview/remediation/40-security-dashboard.md](../../remediation/40-security-dashboard.md). Tiers follow the [00-overall.md](../00-overall.md) roadmap: SEC-31/32 are the P0 failure-limiter re-partition (roadmap item 3), SEC-33/36 ride the P1 residual sweep (item 8), SEC-34 is a P2 Low (item 9), SEC-35 is doc-only. The still-open prior backlog is referenced where a fix should co-locate: **SEC-23** (validator coverage of dashboard/cookie flags) shares the validator pass with SEC-33, **SEC-24** (effective-config view omissions) should absorb the new `MxGateway:Security` keys introduced here, and **SEC-14** (unauthenticated `/metrics`) constrains what the new limiter telemetry may expose.
+
+Repo rules that bind every entry: docs change in the same commit as the source (`CLAUDE.md`), never log or commit secrets (this document references the committed LDAP credential **by location only**, never by value), `TreatWarningsAsErrors=true` (new public members need XML docs), and targeted `dotnet test --filter` runs per task — not the full suite.
+
+## Register
+
+| ID | Sev | Tier | Eff | Dep | Status | Title |
+|----|-----|------|-----|-----|--------|-------|
+| SEC-31 | Medium | P0 | M | — | Not started | Failure limiter partitions on attacker-controlled key id and blocks before verification (lockout DoS) |
+| SEC-32 | Low | P0 | S | SEC-31 | Not started | Failure-limiter LRU is flushable by junk-token spray; token prefix never validated |
+| SEC-33 | Low | P1 | M | — (co-locate SEC-23) | Not started | Any-platform path-rooting acceptance re-opens SEC-01 on Unix; Galaxy `SnapshotCachePath` unvalidated |
+| SEC-34 | Low | P2 | S | — | Not started | Verification cache: expiry outlives TTL; `Invalidate` races in-flight repopulation |
+| SEC-35 | Info | — | S | — | N/A (doc-only note) | Production hard-stops key on the exact `Production` environment name |
+| SEC-36 | Low | P1 | M | cross-repo (`scadaproj/infra/glauth`) | Not started | Committed dev LDAP service-account password: remove from repo and rotate |
+
+---
+
+## SEC-31 — Failure limiter partitions on attacker-controlled key id and blocks before verification   `Medium` · `P0`
+
+**Finding.** `ResolvePeerKey` derives the limiter partition from the **unauthenticated** presented token — `parts[1]` of `mxgw_<keyId>_<secret>` (`src/ZB.MOM.WW.MxGateway.Server/Security/Authorization/GatewayGrpcAuthorizationInterceptor.cs:115-136`) — and `IsBlocked` short-circuits with `ResourceExhausted` *before* `VerifyAsync` runs (`:72-78`). The success-path `Reset` (`:97`) is only reachable after a verification, which a blocked partition never gets. Defaults: 10 failures per 60 s window (`src/ZB.MOM.WW.MxGateway.Server/Configuration/SecurityOptions.cs:51,57`), limiter internals at `src/ZB.MOM.WW.MxGateway.Server/Security/Authorization/ApiKeyFailureLimiter.cs:65-114`.
+
+**Impact.** Key ids are not secrets: they are embedded in every token, shown to every dashboard Viewer, and pushed to anonymous-loopback hub clients in the snapshot's API-key inventory. Any network peer that knows (or enumerates) a key id can send 10 garbage-secret requests per minute and deny that key **indefinitely** — the legitimate client presenting the *correct* secret is rejected before its credential is ever checked, and because rejection precedes verification, the reset that would clear the block is unreachable. The NAT-caveat comment (`GatewayGrpcAuthorizationInterceptor.cs:113-114`) optimized for not locking out co-located clients and instead handed remote attackers a per-key kill switch. Ten packets per minute is all it costs to keep it held down forever.
+
+**Design.** This is a re-partition plus an admission-valve redesign. The limiter exists for two guarantees that any fix must preserve: (G1) online secret-guessing against a key is bounded (~`ApiKeyFailureLimit` per window), and (G2) the failure path does not spend a SQLite read per attempt (failures are never cached by `CachingApiKeyVerifier`, so without the pre-verify short-circuit every guess hits the store). The new guarantee is (G3): no unauthenticated input may indefinitely deny a key to a holder of the correct secret.
+
+*Options considered:*
+
+1. **Pure transport-peer partition** (`context.Peer`, mirroring the dashboard login limiter). Satisfies G3 — an attacker only blocks their own address. Rejected as the sole partition for two reasons: **NAT sharing** — all clients behind one NAT share one budget, so a single fat-fingering (or malicious) neighbor locks the address for every key (the exact scenario the current keying was built to avoid, per `glauth.md`'s NAT caveat); and **IPv6 rotation** — an attacker holding a /64 mints fresh source addresses at will, so the per-key guessing bound (G1) collapses to `limit × addresses`, i.e. effectively unbounded.
+
+2. **Post-verification-only failure counting** (verify first, never block pre-verify). Satisfies G3 by construction — the correct secret always reaches the constant-time compare. Rejected as the sole mechanism because it abandons G2 entirely: every failed guess costs the full library verify (SQLite read + peppered hash + compare), which is precisely the resource the limiter shields (the interceptor comment at `:67-71` states this as the design intent).
+
+3. **Composite `(transport peer, key id)` partition** for the pre-verify check. An attacker's spam only trips (their address × that key); the legitimate holder on any other address is untouched, and co-located NAT clients using *different* keys are untouched. Rejected as the sole mechanism because IPv6 rotation again weakens G1: `limit` guesses per window per minted address.
+
+4. **Chosen: composite partition + per-key-id aggregate, with probe admission instead of absolute blocking.** Two layers over one shared window mechanism:
+   - **Layer 1 — composite `(peer, keyId)` partition**, checked pre-verify as today. Bounds per-address guessing at `ApiKeyFailureLimit`/window and preserves G2 for the common single-source case.
+   - **Layer 2 — per-key-id aggregate**: failures for a (validly-shaped, see SEC-32) key id are also counted across *all* peers. When the aggregate exceeds `ApiKeyFailureAggregateLimit` (new, default 30/window), the key id enters **probe mode**.
+   - **Probe admission**: an over-limit state (either layer) is not an absolute wall — at most one request per `ApiKeyFailureProbeIntervalSeconds` (new, default 5) is admitted through to the real verifier; all others still short-circuit with `ResourceExhausted`. A successful verification resets both the composite partition and the aggregate (making the success-reset path *reachable while throttled*, which is the structural fix for the lockout inversion).
+
+   *Resulting bounds:* a single attacking peer gets `limit`/window + 1 probe per interval (≈ 10 + 12 per minute at defaults) — the same order as the current intent. A distributed/rotating attacker gets at most `ApiKeyFailureAggregateLimit` guesses in the first window before the aggregate trips, then ~1 probe per interval **globally for that key** — which is *stronger* than option 3 and within the same order as today's per-key bound (G1 holds). A legitimate holder presenting the correct secret authenticates immediately from any un-tripped address, and under active spray recovers within a small number of probe intervals (it competes for probe slots, but a win resets all state; the attacker must then re-trip the aggregate from scratch). Residual accepted: a high-rate distributed sprayer can *delay* (not deny) the legitimate holder — but that now requires sustained, metrically-visible attack traffic instead of 10 packets/min, and G3's "indefinitely" is eliminated.
+
+*Also rejected:* trusting the key id after a cheap local check (any format check is attacker-satisfiable — the design must assume key ids are public); allowlisting known key ids by querying the store pre-verify (reintroduces the per-attempt store read G2 forbids).
+
+*Telemetry note:* add throttle counters tagged only by stage (`peer`/`aggregate`) — never by key id or peer address (cardinality per the SEC-20 precedent, and `/metrics` is still unauthenticated per open **SEC-14**, so counters must carry no key material). When prior-cycle **SEC-24** (effective-config view) is picked up, the new `MxGateway:Security` keys added here belong in that projection.
+
+**Implementation.**
+
+1. `src/ZB.MOM.WW.MxGateway.Server/Configuration/SecurityOptions.cs`: add `int ApiKeyFailureAggregateLimit { get; init; } = 30;` (0 disables the aggregate layer) and `int ApiKeyFailureProbeIntervalSeconds { get; init; } = 5;` (0 disables probe admission → absolute block, documented as not recommended). Full XML docs (warnings-as-errors).
+2. `src/ZB.MOM.WW.MxGateway.Server/Configuration/GatewayOptionsValidator.cs` `ValidateSecurity`: `AddIfNegative` for both new keys, with messages naming the config paths (`MxGateway:Security:ApiKeyFailureAggregateLimit`, `...:ApiKeyFailureProbeIntervalSeconds`).
+3. `src/ZB.MOM.WW.MxGateway.Server/Security/Authorization/ApiKeyFailureLimiter.cs`: rework the public surface from `IsBlocked/RecordFailure/Reset(string peer)` to a partition-pair API, e.g. `ThrottleDecision Check(ApiKeyThrottlePartition partition)` / `RecordFailure(partition)` / `Reset(partition)` where `ApiKeyThrottlePartition` carries `TransportPeer` (required) and `KeyId` (optional, only when the token is validly shaped per SEC-32). Internal state: `_partitions` keyed `"{peer}|{keyId}"` (or `"{peer}"` fallback) and `_aggregates` keyed by key id, both reusing the existing sliding-window `PeerState` plus a `NextProbeAtTicks` field; probe slots are granted under the per-state lock. Keep the `TimeProvider` seam and the internal test constructor. Rewrite the class remarks — the current NAT rationale (`:13-26`) describes the defective keying.
+4. `src/ZB.MOM.WW.MxGateway.Server/Security/Authorization/GatewayGrpcAuthorizationInterceptor.cs`: replace `ResolvePeerKey` with a `ResolveThrottlePartition(authorizationHeader, context)` that returns the pair (with the SEC-32 shape check); thread the probe-admission decision through `AuthenticateAndAuthorizeAsync` (a probe-admitted request proceeds to `VerifyAsync`; a throttled one throws `ResourceExhausted` with the existing opaque message). `RecordFailure`/`Reset` take the partition pair. Update the comment block at `:67-78,112-114`.
+5. Optional but recommended: a `mxgateway.auth.throttled` counter in `Metrics/GatewayMetrics.cs` tagged `stage=peer|aggregate` only.
+6. Docs, same commit: rewrite `docs/GatewayConfiguration.md:282-284` (the three existing limiter rows describe key-id keying and "resets on success" semantics that change here) and add rows for the two new keys; update the failure-limiter paragraph in `docs/Authentication.md` (hot-path section, ~`:85-115`) to describe the two-layer partition and probe admission.
+7. Tests: new `src/ZB.MOM.WW.MxGateway.Tests/Security/Authorization/ApiKeyFailureLimiterTests.cs` (window pruning, composite-vs-aggregate trip points, probe cadence, reset clears both layers); update the two existing limiter-related interceptor tests (`GatewayGrpcAuthorizationInterceptorTests.cs:363-460`) for the new API.
+
+New tests (names are the contract):
+- `AttackerSpamOnVictimKeyId_FromDifferentPeer_DoesNotBlockLegitimateHolderPresentingCorrectSecret` — flood failures for key id K from peer A until blocked; a call from peer B with the correct secret for K verifies successfully on the first attempt (assert the verifier **was** called and the RPC succeeded).
+- `BruteForceBound_StillEnforcedPerAttackingPeer` — same peer, same key id: attempt `limit`+1 wrong secrets inside the window; assert the final attempt maps to `ResourceExhausted` **before** the verifier is called (reuse `CountingFailureVerifier`).
+- `AggregateSpray_AcrossManyPeers_TripsPerKeyProbeMode` — failures for one key id from > `AggregateLimit` distinct peers; assert subsequent attempts from a fresh peer are probe-limited (at most one verifier call per probe interval, others `ResourceExhausted`).
+- `CorrectSecret_DuringProbeMode_AuthenticatesViaProbeSlotAndResets` — while throttled, advance the fake `TimeProvider` past the probe interval; the correct secret is admitted, succeeds, and fully resets the state (next wrong attempt is `Unauthenticated`, not `ResourceExhausted`).
+
+**Verification.** `dotnet build src/ZB.MOM.WW.MxGateway.Server` then:
+```
+dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter "FullyQualifiedName~ApiKeyFailureLimiter"
+dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter "FullyQualifiedName~GatewayGrpcAuthorizationInterceptor"
+dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter "FullyQualifiedName~GatewayOptionsValidator"
+```
+
+---
+
+## SEC-32 — Failure-limiter LRU is flushable by junk-token spray; token prefix never validated   `Low` · `P0` · depends on SEC-31
+
+**Finding.** The tracked-peer map evicts the least-recently-active entry once it exceeds `ApiKeyFailureTrackedPeers` (default 4096) — `src/ZB.MOM.WW.MxGateway.Server/Security/Authorization/ApiKeyFailureLimiter.cs:124-148` — and the partition key is minted from attacker-chosen token text with **no prefix check**: any `a_b_c`-shaped garbage creates a fresh `key:b` partition (`GatewayGrpcAuthorizationInterceptor.cs:127-132`; the `mxgw` literal is never compared).
+
+**Impact.** A guesser interleaves real guesses with ~4096 unique throwaway tokens to evict their own `PeerState` (or a SEC-31 victim's blocked state) and reset the window, resuming past the 10-per-minute bound at a cost of ~4096 requests per flush cycle. The documented guarantee ("a spray of unique peer keys cannot grow memory without limit", `SecurityOptions.cs:59-64`) is true but silently trades away the *blocking* guarantee it exists to serve.
+
+**Design.** Three reinforcing changes, landed inside the SEC-31 rework (same types, same tests, one commit train):
+
+1. **Validate token shape before minting a key-id partition.** Only a token whose first segment is the literal `mxgw`, with ≥ 3 non-empty `_`-segments and a key id within a sane length cap (≤ 64 chars — generated key ids are far shorter), gets a `KeyId` on its throttle partition. Everything else falls to the transport-peer fallback partition — so a junk spray from one address accumulates in **one** partition (which then throttles that address wholesale) instead of minting thousands.
+2. **Cap key-id partitions per transport peer.** SEC-31's composite keying already binds every partition to the sender's address, but a single address can still mint one partition per unique `mxgw`-shaped key id it invents. Cap distinct key-id partitions per transport peer at a small constant (32; not config — there is no legitimate reason for one address to fail against dozens of distinct keys inside one window); overflow collapses into that address's fallback partition. This bounds the total partitions one attacker can mint to 33 per source address.
+3. **Eviction never removes load-bearing state.** `EvictIfOverCapacity` preference order: (a) entries whose windows are empty (fully expired) first; (b) never evict an entry that is currently over-limit / in probe mode — such entries decay naturally within `ApiKeyFailureWindowSeconds`, so the exemption cannot pin memory indefinitely; (c) otherwise least-recently-active. Allow a documented transient overshoot of `ApiKeyFailureTrackedPeers` for non-evictable over-limit entries with a hard ceiling of 2× the cap (beyond which oldest-blocked is evicted anyway — availability of the bound beats perfection of the block under a distributed attack that large).
+
+*Rejected alternatives:* validating the key id against the store pre-verify (reintroduces the per-attempt SQLite read); a cryptographic/entropy check on the key-id segment (attacker-satisfiable, adds nothing over the length/prefix check); simply raising `ApiKeyFailureTrackedPeers` (changes the flush price, not the defect).
+
+**Implementation.**
+
+1. `GatewayGrpcAuthorizationInterceptor.ResolveThrottlePartition` (from SEC-31 step 4): add the `mxgw` prefix + segment + key-id-length checks; on failure return a partition with `KeyId = null`.
+2. `ApiKeyFailureLimiter`: per-transport-peer key-id partition counter (cap 32, collapse to fallback); eviction policy per design point 3 (`EvictIfOverCapacity` rewrite; over-limit detection reuses the same window state the `Check` path computes).
+3. Update the limiter remarks and `SecurityOptions.ApiKeyFailureTrackedPeers` XML doc (the "cannot grow memory without limit" sentence gains the "and cannot be flushed" half plus the 2× transient note).
+4. Docs, same commit: `docs/GatewayConfiguration.md` `ApiKeyFailureTrackedPeers` row — describe the eviction preference and the flush resistance.
+5. Tests (in `ApiKeyFailureLimiterTests` / `GatewayGrpcAuthorizationInterceptorTests`):
+   - `NonMxgwToken_FallsBackToTransportPeerPartition` — junk tokens of varied shapes all land on the sender's fallback partition (assert one tracked entry).
+   - `JunkTokenSpray_DoesNotEvictBlockedEntry` — trip a block for (peer A, key K); spray 2× `TrackedPeers` unique tokens from other peers; assert (A, K) is still blocked.
+   - `UniqueMxgwKeyIdSpray_FromOnePeer_CollapsesAtPerPeerCap` — 1000 unique `mxgw`-shaped key ids from one address mint ≤ 32 + 1 partitions and end up throttling the address's fallback partition.
+   - `Eviction_PrefersExpiredWindows` — with the map at capacity, a new failure evicts an entry whose window has fully expired, not an active one.
+
+**Verification.** Same filters as SEC-31 (`~ApiKeyFailureLimiter`, `~GatewayGrpcAuthorizationInterceptor`) after `dotnet build src/ZB.MOM.WW.MxGateway.Server`.
+
+---
+
+## SEC-33 — Any-platform path-rooting acceptance re-opens SEC-01 on Unix; Galaxy `SnapshotCachePath` unvalidated   `Low` · `P1` · co-locate with open SEC-23
+
+**Finding.** `IsRootedForAnyPlatform` (`src/ZB.MOM.WW.MxGateway.Server/Configuration/GatewayOptionsValidator.cs:541-560`) deliberately passes Windows drive-qualified/UNC paths on Unix so the shipped `appsettings.json` validates cross-platform. But the validator runs in the same process that then *uses* the path: on Unix, `C:\ProgramData\MxGateway\gateway-auth.db` (`appsettings.json:17`) contains no Unix separator, SQLite treats it as one relative filename, and the credential store lands in the CWD — the original SEC-01 mechanism, now behind a validator message promising it "never lands in the launch working directory" (`:112`). Evidence at HEAD: a stray auth DB dated 2026-07-09 (post-SEC-01-fix) under `src/ZB.MOM.WW.MxGateway.Tests/bin/Debug/net10.0/` with the literal Windows path as its filename, invisible to the hygiene test's bin/obj filter (`src/ZB.MOM.WW.MxGateway.Tests/ProjectStructure/GatewayTreeHygieneTests.cs:30-35`). Additionally Galaxy `SnapshotCachePath` (`appsettings.json:80`) gets **no** rooting validation: the shared `ZB.MOM.WW.GalaxyRepository` package binds it (`GatewayApplication.cs:103`) and deliberately ships no validator — per `A2-galaxyrepository-adoption-handoff.md:157-160` the gateway was supposed to own that validation, and doesn't.
+
+**Impact.** Any non-Windows run (dev box, tests, future container) that binds the shipped config silently writes the API-key credential store — and the Galaxy snapshot — as junk-named files relative to whatever the CWD happens to be. The validator's guarantee is false on the platform where it matters; the hygiene test cannot see the failure because the artifacts land in gitignored build output.
+
+**Design.** Make rooting **host-meaningful** and stop shipping foreign-platform literals, instead of teaching the validator to bless paths the process cannot use:
+
+1. **Validator checks the running OS.** `AddIfNotRooted` uses plain `Path.IsPathRooted` (current platform); delete `IsRootedForAnyPlatform`. A Windows drive path on Unix becomes a fail-fast startup error naming the offending key — exactly how every other misconfiguration surfaces. The "validate prod config offline on macOS" rationale in the doc-comment dissolves with change 2, and no offline-validation tooling exists that would need the any-platform mode.
+2. **Remove the Windows literals from `appsettings.json`.** Drop `Authentication:SqlitePath` (`:17`) and `Galaxy:SnapshotCachePath` (`:80`); the `SpecialFolder.CommonApplicationData`-derived code defaults (SEC-01's own mechanism, `Configuration/AuthenticationOptions.cs:16-19`) resolve to the **identical** `C:\ProgramData\MxGateway\...` on Windows and a platform-appropriate path elsewhere. Deployed hosts are unaffected: production config rides NSSM environment variables and the Windows code default is byte-identical to the removed literal. For Galaxy, add a gateway-side `PostConfigure` default (`Path.Combine(CommonApplicationData, "MxGateway", "galaxy-snapshot.json")`) applied when the bound value is blank.
+3. **Gateway-side Galaxy options validation.** New `Configuration/GalaxyRepositoryOptionsValidator.cs` registered as `IValidateOptions<GalaxyRepositoryOptions>` (the package type — the lib binds only, by design): when `PersistSnapshot` is true, `SnapshotCachePath` must be non-blank, a valid path, and rooted on the current OS (reuse the `AddIfInvalidPath`/`AddIfNotRooted` helpers — promote them to a shared internal helper if the new validator can't reach them).
+4. **Root-cause the stray file.** Identify the test/tool run that bound the shipped `appsettings.json` and materialized the DB under the test bin dir; pin it to a temp path (the `TempDatabaseDirectory` helper exists in `Tests/Security/Authentication/`). After changes 1–2 the failure mode is loud (validation error) rather than silent, and on macOS the code default may point at an unwritable `/usr/share/...` — any test exercising the real startup path must override `SqlitePath`. Delete the stray gitignored file.
+
+*Rejected alternatives:* auto-rooting/rewriting foreign-platform paths to a host default (silent relocation of a credential store — worse than a boot error, per the SEC-01 design note at `GatewayOptionsValidator.cs:515-521`); keeping `IsRootedForAnyPlatform` in the validator and adding a second use-site runtime check (two checks to keep aligned, and the validator's promise stays false); extending the hygiene test to scan bin/obj (treats the symptom; build output is transient).
+
+**Implementation.**
+
+1. `Configuration/GatewayOptionsValidator.cs`: `AddIfNotRooted` → `Path.IsPathRooted`; delete `IsRootedForAnyPlatform` (`:533-560`) and its doc comment.
+2. `src/ZB.MOM.WW.MxGateway.Server/appsettings.json`: remove the `SqlitePath` and `SnapshotCachePath` lines (code defaults take over).
+3. New `Configuration/GalaxyRepositoryOptionsValidator.cs` (+ registration next to `AddZbGalaxyRepository` in `GatewayApplication.cs:99-103`) and a `PostConfigure<GalaxyRepositoryOptions>` default for `SnapshotCachePath`.
+4. Find and fix the offending test per design point 4; `rm` the stray `.../bin/Debug/net10.0/C:\ProgramData\MxGateway\gateway-auth.db`.
+5. Docs, same commit: `docs/GatewayConfiguration.md` — `SqlitePath` / `SnapshotCachePath` rows now document the derived per-OS default and the rooted-on-this-host rule; `docs/Authentication.md` default-path sentence if it names the Windows literal.
+6. Tests: `Tests/Configuration/GatewayOptionsValidatorTests.cs` — on the running platform, a foreign-platform-rooted path fails (`C:\x\y.db` on Unix; assert via the injected message), a host-rooted path passes, a bare filename fails; new `GalaxyRepositoryOptionsValidatorTests` — `PersistSnapshot=true` + blank/relative path → invalid, disabled persistence skips the check, blank path gets the PostConfigure default.
+
+Note for the executor: this pass touches the same `ValidateDashboard`-adjacent validator file as still-open **SEC-23** (cookie/`__Host-`/`AutoLoginUser` coverage) — if SEC-23 is picked up this cycle, land them as one validator commit.
+
+**Verification.** `dotnet build src/ZB.MOM.WW.MxGateway.Server` (on macOS this also proves the shipped config still boots validation-clean after the literal removal) then:
+```
+dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter "FullyQualifiedName~GatewayOptionsValidator"
+dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter "FullyQualifiedName~GalaxyRepositoryOptionsValidator"
+dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter "FullyQualifiedName~GatewayTreeHygiene"
+```
+Post-run, verify no new `C:\*` file exists under any `bin/` (manual `find src -name 'C:*'`).
+
+---
+
+## SEC-34 — Verification cache: expiry outlives TTL; `Invalidate` races in-flight repopulation   `Low` · `P2`
+
+**Finding.** Three bounded staleness windows in `src/ZB.MOM.WW.MxGateway.Server/Security/Authentication/CachingApiKeyVerifier.cs`: (1) a CLI-issued revoke (separate process) keeps authenticating ≤ TTL — documented (`:41-47`); (2) a key whose `ExpiresUtc` passes while cached keeps working ≤ 15 s past expiry (`SecurityOptions.cs:21`) — expiry is enforced by the inner library verifier, which a cache hit never reaches, and this case is **not** in the documented backstop rationale; (3) race: a `VerifyAsync` that passed the inner verifier just before a dashboard revoke can `_cache.Set` (`:110-117`) *after* `Invalidate(keyId)` ran (`:123-137` — no generation check), re-caching the revoked identity for a full TTL despite the "gateway-initiated mutations take effect immediately" contract (`:11-13`).
+
+**Impact.** All three are bounded (≤ 15 s; ~30 s worst case for the race), so exposure is small — but (2) silently contradicts the SEC-10 expiry feature's semantics, and (3) contradicts the class's own stated contract, which is the kind of gap the next review flags as "documented behavior is false".
+
+**Design.**
+
+- **Expiry (window 2): eliminate, don't document.** The shared `ApiKeyIdentity` (0.1.4) carries `ExpiresUtc`; when caching a success, cap the entry lifetime at the key's expiry: `AbsoluteExpiration = min(now + ttl, ExpiresUtc)` (skip caching entirely if already ≤ now). A cached hit can then never outlive the key. Confirm during implementation that the library verifier populates `ExpiresUtc` on the returned identity; if it does not, fall back to documenting the ≤ TTL window in the remarks and `docs/Authentication.md` and file a donor-library ask.
+- **Invalidate race (window 3): per-key generation check.** `ConcurrentDictionary<string, long> _generations`; `Invalidate(keyId)` increments the generation **before** evicting cache keys. `VerifyAsync` parses the key id from the token up front (same split the interceptor does — cheap, no store access), snapshots `g0` before calling the inner verifier, and after a success only `Set`s when the generation still equals `g0` — then re-reads the generation after the `Set` and self-evicts if it moved (bump-before-evict + set-then-recheck closes the remaining interleaving). Unparseable tokens skip caching already (`TryComputeCacheKey`).
+- **CLI window (1): accept and keep documented** — cross-process invalidation is out of scope by design; the TTL is the backstop and the remarks already say so.
+
+*Rejected alternatives:* a lock around verify+set (serializes the auth hot path the cache exists to speed up); switching to a distributed/shared cache (massive machinery for a 15 s window); "accept and document" for the race (viable and cheap, but the generation counter is ~20 lines and makes the stated contract true — prefer honest code over honest caveats).
+
+**Implementation.**
+
+1. `Security/Authentication/CachingApiKeyVerifier.cs`: expiry-capped `MemoryCacheEntryOptions` in `VerifyAsync` (`:110-117`); `_generations` + snapshot/recheck logic; `Invalidate` bumps generation first (`:123-137`). Update the remarks (`:29-47`): add the expiry cap and the generation mechanism; keep the CLI-window caveat.
+2. Docs, same commit: `docs/Authentication.md` hot-path caching section (~`:92-105`) — one sentence each for the expiry cap and the invalidate-vs-in-flight guarantee.
+3. Tests, `src/ZB.MOM.WW.MxGateway.Tests/Security/Authentication/CachingApiKeyVerifierTests.cs`:
+   - `CacheEntry_DoesNotOutliveKeyExpiry` — identity expiring 5 s from now with a 15 s TTL: hit at +4 s served from cache, attempt at +6 s reaches the inner verifier.
+   - `AlreadyExpiredIdentity_IsNotCached` — inner returns success with `ExpiresUtc` ≤ now (defensive); nothing is cached.
+   - `Invalidate_DuringInFlightVerification_DiscardsStaleRepopulation` — inner verifier gated on a `TaskCompletionSource`; call `Invalidate(keyId)` while `VerifyAsync` is awaiting inner, then release; assert the follow-up `VerifyAsync` calls the inner verifier again (no stale cache hit).
+
+**Verification.** `dotnet build src/ZB.MOM.WW.MxGateway.Server` then:
+```
+dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter "FullyQualifiedName~CachingApiKeyVerifier"
+```
+
+---
+
+## SEC-35 — Production hard-stops key on the exact `Production` environment name   `Info` · `—` (N/A: doc-only)
+
+**Finding.** The `DisableLogin` and plaintext-LDAP guards fire only when `IHostEnvironment.IsProduction()` (`src/ZB.MOM.WW.MxGateway.Server/Configuration/GatewayOptionsValidator.cs:23-27,161-165,314-318`) — i.e. `ASPNETCORE_ENVIRONMENT` unset (defaults to Production; the NSSM-deployed hosts are covered) or exactly `Production`. A host launched as `Staging`, `Prod`, or any custom name silently keeps the permissive dev posture. The fallback-environment wiring (`Configuration/GatewayConfigurationServiceCollectionExtensions.cs:24,38-51`) is correct.
+
+**Impact.** None at current deployments; a latent surprise for a future operator who invents an environment name and assumes the guards travel with it.
+
+**Design.** N/A for code, by deliberate acceptance: the review itself rates this acceptable-as-designed, ASP.NET Core's environment-name semantics are the platform convention, and inverting to "not Development ⇒ production-like" would hard-stop legitimate permissive staging rigs (the shared dev GLAuth is plaintext-only, so a `Staging` host pointed at it would refuse to boot) — a behavior change with real downside for an Info-severity conventions note. The remediation is the documentation contract the review asked for.
+
+**Implementation.** `docs/GatewayConfiguration.md` (environment/hard-stop area, near `:244-254`): one short paragraph stating that the two production hard-stops key on `IHostEnvironment.IsProduction()` — the exact `Production` name or an unset `ASPNETCORE_ENVIRONMENT` — and that any other environment name retains the permissive dev posture, so production-like deployments must use the literal `Production`. Same-commit rider on whichever SEC-33/SEC-36 change touches that doc first.
+
+**Verification.** Doc-only; no build. Cross-read against `GatewayOptionsValidator.cs:23-27`.
+
+---
+
+## SEC-36 — Committed dev LDAP service-account password: remove from repo and rotate   `Low` · `P1` · cross-repo dependency
+
+**Finding.** The SEC-06 remediation shipped the production transport hard-stop and the env-var override documentation, but the dev LDAP service-account credential itself is still committed at HEAD: `src/ZB.MOM.WW.MxGateway.Server/appsettings.json:29` (`Ldap:ServiceAccountPassword`), with the same value repeated in `glauth.md`'s samples (`:33,65,102-103,135-136,245`) and acknowledged as "should be rotated" by `docs/GatewayConfiguration.md:248`. (Value intentionally not reproduced here.)
+
+**Impact.** As long as the shared GLAuth instance (`10.100.0.35:3893`) honors this credential, the repo — and its full git history — discloses a live directory service account with LDAP search capability over `dc=zb,dc=local`. Removal alone is insufficient: the value is permanently recoverable from history, so **rotation is the load-bearing half** of the fix.
+
+**Design.** Rotate first, then remove; give dev a supported secret channel so the removal doesn't break the local login flow.
+
+- **Rotation (cross-repo).** The GLAuth source of truth is `scadaproj/infra/glauth/` (per `glauth.md`; note `scadaproj` is a shared monorepo — stage explicit paths only). Generate a new service-account secret there, redeploy GLAuth on `10.100.0.35`, and record the rotation (date, not value) in `glauth.md`. The new value must never be committed to *this* repo in any file — `glauth.md`'s samples switch to a `<service-account-password>` placeholder with a pointer at the source of truth.
+- **Removal.** Delete the `ServiceAccountPassword` line from `appsettings.json`. The validator already fails a blank password when `Ldap.Enabled=true` (`GatewayOptionsValidator.cs:134-137`), so a host without the secret fails fast at startup instead of failing binds at runtime — extend that validation message to name the two supported channels.
+- **Where the dev secret lives instead.** Two supported channels, both flowing through the existing `MxGateway:Ldap:ServiceAccountPassword` binding:
+  1. **Deployed hosts (unchanged):** the env-var override `MxGateway__Ldap__ServiceAccountPassword` set in the NSSM service environment — already the documented production mechanism (`docs/GatewayConfiguration.md:248`) and already how deployed config works.
+  2. **Dev boxes (new):** .NET user-secrets — add `<UserSecretsId>` to `ZB.MOM.WW.MxGateway.Server.csproj`; `WebApplicationBuilder` loads user-secrets automatically in Development, so `dotnet user-secrets set "MxGateway:Ldap:ServiceAccountPassword" <value>` (value obtained from the GLAuth source of truth) is a one-time per-machine step. Secrets live under the user profile, outside the tree.
+- **Cutover order** (avoids a broken window): set the env var on the deployed hosts with the *new* value → rotate GLAuth → verify dashboard login on the deployed hosts → land the repo change (removal + docs) → devs set user-secrets on next pull (startup error message tells them exactly what to do). Check the second deployment (`wonder-app-vd03`) for an LDAP binding before assuming it needs the var (its dashboard is disabled; if `Ldap.Enabled=false` there, nothing to do).
+
+*Rejected alternatives:* committing a placeholder string instead of deleting the key (passes blank-validation with a wrong value and fails later at bind time — a silent misconfiguration where fail-fast is available); `appsettings.Development.json` (still a committed file — same defect relocated); rotating prod-only and keeping the dev credential committed (the finding *is* the committed live dev credential); encrypting the value in config (key-management theater for a dev secret with a working out-of-band channel).
+
+**Implementation.**
+
+1. *(cross-repo, first)* Rotate in `scadaproj/infra/glauth/` and redeploy GLAuth; pre-stage `MxGateway__Ldap__ServiceAccountPassword` in the NSSM environment on `10.100.0.48` (and `wonder-app-vd03` only if LDAP is enabled there); verify dashboard login post-rotation.
+2. `src/ZB.MOM.WW.MxGateway.Server/appsettings.json:29`: delete the `ServiceAccountPassword` entry.
+3. `src/ZB.MOM.WW.MxGateway.Server/ZB.MOM.WW.MxGateway.Server.csproj`: add `<UserSecretsId>mxaccessgw-server</UserSecretsId>`.
+4. `Configuration/GatewayOptionsValidator.cs:134-137`: extend the blank-password message to point at user-secrets (dev) and the env-var override (deployed).
+5. Docs, same commit as 2–4: `docs/GatewayConfiguration.md:248` — replace the "dev-only committed value / should be rotated" wording with the two channels and the rotation note; `glauth.md` — placeholder in all samples (`:33,65,102-103,135-136,245`), rotation recorded, pointer to `scadaproj/infra/glauth/`; `CLAUDE.md`'s `glauth.md` summary line if it references the credential.
+6. Repo scrub check: `git grep` for the old value across tracked files must return nothing (run manually — do not encode the secret into a test).
+
+**Verification.** `dotnet build src/ZB.MOM.WW.MxGateway.Server`; then
+```
+dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter "FullyQualifiedName~GatewayOptionsValidator"
+```
+(asserts the blank-password validation still fires with the updated message). Manual: with user-secrets set on the dev box, `dotnet run --project src/ZB.MOM.WW.MxGateway.Server/...` and a dashboard `/login` as `multi-role` succeeds against the rotated GLAuth; the deployed-host login re-check from step 1 counts as the production verification. Live-LDAP integration tests (`MXGATEWAY_RUN_LIVE_LDAP_TESTS=1`) only where the GLAuth instance is reachable; otherwise document skipped per the testing matrix.

archreview/2026-07-12/remediation/50-clients.md

@@ -0,0 +1,251 @@
+# Language Clients — Remediation Design & Implementation (2026-07-12 cycle)
+
+Source review: [50-clients.md](../50-clients.md) · Generated: 2026-07-13 · Baseline: `main @ 4f5371f`
+
+This document turns the 2026-07-12 Language Clients re-review's **new** findings (CLI-35..CLI-45) into buildable remediation entries. Tier assignments come from the [overall roadmap](../00-overall.md). Prior-cycle findings CLI-01..CLI-34 remain tracked in `archreview/remediation/50-clients.md`; the one prior finding this cycle re-activates is **CLI-08**, which is closed by CLI-38 below.
+
+Operating constraints carried from prior work:
+
+- **Java builds locally now**: `JAVA_HOME=/opt/homebrew/opt/openjdk@17 gradle test` from `clients/java`. The Gradle build regenerates the tracked `clients/java/**/src/main/generated/**/MxaccessGateway.java` with spurious protobuf-version churn (~64k lines) — **revert it (`git checkout -- clients/java/**/generated`) after every Java build in this plan, because no `.proto` changes here**.
+- **No `.proto` changes anywhere in this plan** — no contracts regen, no Python `grpcio-tools` regen, no vendored-Rust-proto refresh.
+- Per-language verification follows CLAUDE.md's Source Update Workflow table: `gofmt` + `go build ./...` + `go test ./...`; `cargo fmt` + `cargo check --workspace` + `cargo test --workspace` + `cargo clippy --all-targets -- -D warnings`; `python -m pytest`; `gradle test`; `dotnet build clients/dotnet/ZB.MOM.WW.MxGateway.Client.slnx` + tests.
+- **Cross-language behavior findings (CLI-37/38/41/45) are conformance exercises, not five independent fixes**: each Design section below defines one canonical behavior first (citing the wire contract), then per-language implementation. Where the behavior is specified in `docs/CrossLanguageSmokeMatrix.md` or `docs/ClientLibrariesDesign.md`, those docs change in the same commit (CLAUDE.md same-commit rule).
+- Shared behavior fixtures live under `clients/proto/fixtures/behavior/` (manifest: `clients/proto/fixtures/behavior/manifest.json`, described by `docs/ClientBehaviorFixtures.md`) — new conformance cases land there so all five suites lock in the same bytes.
+
+## Finding register
+
+| ID | Sev | Tier | Eff | Dep | Status | Title |
+|----|-----|------|-----|-----|--------|-------|
+| CLI-35 | Medium | P0 | S | — | Not started | Python CLI `stream-events` crashes on a ReplayGap |
+| CLI-36 | Medium | P0 | S | — | Not started | Go CLI `stream-events` silently destroys the ReplayGap signal |
+| CLI-37 | Medium | P1 | M | CLI-38 | Not started | Status-array validation must branch on `category` per the proto contract (4-vs-1 divergence) |
+| CLI-38 | Medium | P1 | S | — | Not started | Align .NET/Go/Java on `hresult < 0` — lands prior CLI-08 and cures the design-doc drift |
+| CLI-39 | Medium | P1 | S | CLI-35..38, CLI-45 | Not started | Bump client versions off the already-published 0.1.2 before the next publish; add registry-collision guard |
+| CLI-40 | Low | — | M | — | Not started | Port the exact-secret credential scrub to Rust/Java/.NET |
+| CLI-41 | Low | — | M | — | Not started | Uniform malformed-reply contract for AuthenticateUser/ArchestrAUserToId/AddBufferedItem |
+| CLI-42 | Low | P1 | S | — | Not started | Document the vendored Rust proto layout (CLI-02's missing doc half) |
+| CLI-43 | Low | — | S | — | Not started | Java style guide still prescribes "Java 21 preferred" |
+| CLI-44 | Low | — | S | — | Not started | Go event goroutine can mislabel a genuine terminal error as `ErrSlowConsumer` |
+| CLI-45 | Low | P1 | M | — | Not started | Standardize CLI credential env-var name and fail fast on missing/empty passwords |
+
+Cross-domain dependencies: **CLI-35/CLI-36 pair with GWC-25** (gateway emits `oldest_available_sequence = 0` on an empty replay ring — the server-side half of the same reconnect story; the CLI fixes here are independently landable but the end-to-end resume walk in the smoke matrix needs both). **CLI-39 pairs with the publishing process** (`scripts/pack-clients.ps1`, `scripts/tag-go-module.ps1`, Gitea package registry).
+
+---
+
+## CLI-35 — Python CLI `stream-events` crashes on a ReplayGap   `Medium` · `P0`
+
+**Finding.** `Session.stream_events()` yields `pb.MxEvent | ReplayGap` since the CLI-15 library work (`clients/python/src/zb_mom_ww_mxgateway/session.py:816-870`; `ReplayGap` dataclass at `clients/python/src/zb_mom_ww_mxgateway/events.py:11`), but the CLI's `_stream_events` (`clients/python/src/zb_mom_ww_mxgateway_cli/commands.py:1098-1106`) feeds every collected item into `_message_dict` (`commands.py:1627-1632`), which calls `google.protobuf.json_format.MessageToDict`. `ReplayGap` is a plain dataclass, not a proto message, so `MessageToDict` raises and the command exits with an error — after having already consumed the stream. No test covers it (`grep -i replay clients/python/tests/test_cli.py` → nothing).
+
+**Impact.** The operator tool crashes on exactly the resume-after-outage path where it is most needed. Detach-grace and replay retention are on by default, so any `mxgw-py stream-events --after-worker-sequence` resume that outlived the replay ring hits this. P0 per the roadmap (item 1, alongside GWC-25).
+
+**Design.** Branch on `isinstance(item, ReplayGap)` in the CLI's row formatter and emit the same distinct JSON row the Rust CLI already prints (`clients/rust/crates/mxgw-cli/src/main.rs:1019-1035`): `{"replayGap": {"requestedAfterSequence": <n>, "oldestAvailableSequence": <n>}}` — camelCase, matching `_message_dict`'s `preserving_proto_field_name=False` output for normal events, so the JSON stream stays uniform and the cross-language e2e matrix can compare rows across CLIs. The gap is rendered, never dropped and never re-synthesized into an event (no-synthesized-events invariant). Rejected alternatives: (a) yielding the raw sentinel `MxEvent` from the CLI path — would require bypassing the typed library surface the CLI already uses and reintroduces the "gap looks like a data change" hazard the library deliberately closed; (b) filtering gaps out — silent loss, precisely CLI-36's defect.
+
+**Implementation.**
+1. `clients/python/src/zb_mom_ww_mxgateway_cli/commands.py`: import `ReplayGap` from `zb_mom_ww_mxgateway.events`. Add a small row helper, e.g. `_event_row(item)` → `_message_dict(item)` for proto messages, and for `ReplayGap` return `{"replayGap": {"requestedAfterSequence": item.requested_after_sequence, "oldestAvailableSequence": item.oldest_available_sequence}}`.
+2. `_stream_events` (`commands.py:1106`): change `{"events": [_message_dict(event) for event in events]}` to use `_event_row`.
+3. New test in `clients/python/tests/test_cli.py`: `test_stream_events_renders_replay_gap` — monkeypatch `GatewayClient.connect` with the existing fake-connect pattern (see `test_cli.py:21-54`); the fake session's `stream_events` yields a `ReplayGap(requested_after_sequence=7, oldest_available_sequence=42)` followed by one normal `pb.MxEvent`; invoke `stream-events` through `CliRunner` and assert exit code 0, the output JSON contains one `replayGap` row with both sequences, and the normal event row follows it. This is the regression the review found missing.
+4. Docs: extend the ReplayGap section of `docs/CrossLanguageSmokeMatrix.md:33-39` with a per-CLI rendering row (Python: `replayGap` JSON row — same shape as Rust) in the same commit.
+
+**Verification.** From `clients/python`: `python -m pytest` (targeted first: `python -m pytest tests/test_cli.py -k replay_gap`). No proto change → no regen, no pinned `grpcio-tools` concern.
+
+---
+
+## CLI-36 — Go CLI `stream-events` silently destroys the ReplayGap signal   `Medium` · `P0`
+
+**Finding.** The Go library deliberately clears `EventResult.Event` on a gap and populates `EventResult.ReplayGap` (`clients/go/mxgateway/session.go:1036-1042`), but the CLI loop (`clients/go/cmd/mxgw-go/main.go:969-983`) only checks `result.Err` and then formats `result.Event`: JSON mode marshals a nil `*MxEvent` (empty object), text mode prints `0 MX_EVENT_FAMILY_UNSPECIFIED`. The gap's cursors — `requested_after_sequence` and `oldest_available_sequence`, the exact data an operator needs to resume — are discarded.
+
+**Impact.** The typed signal the library was specifically built to preserve is destroyed one layer up; an operator resuming after an outage sees a meaningless zero row instead of the resume cursor. P0 per the roadmap (item 1, alongside GWC-25 and CLI-35).
+
+**Design.** Add an `if result.IsReplayGap()` branch to the CLI loop that renders a dedicated gap row, conforming to the Rust CLI's canonical rendering (`mxgw-cli/src/main.rs:1019-1035`): text mode prints `REPLAY_GAP requested_after=<n> oldest_available=<n>`; JSON mode prints `{"replayGap": {"requestedAfterSequence": <n>, "oldestAvailableSequence": <n>}}` (marshal `result.ReplayGap` with `protojson` under a `replayGap` key, which yields exactly those camelCase fields). The gap row counts toward `-limit` like any other emitted row, mirroring Rust's `events` array accounting. Rejected alternative: printing the raw sentinel event as .NET/Java CLIs do — the Go library intentionally clears `Event`, so there is no sentinel left to print; reconstructing one would synthesize an event.
+
+**Implementation.**
+1. `clients/go/cmd/mxgw-go/main.go` `runStreamEvents` (loop at `:969-983`): after the `result.Err` check, insert:
+   - `if result.IsReplayGap()` → JSON: wrap `protojson.Marshal(result.ReplayGap)` as `{"replayGap":<bytes>}` (or a tiny struct with a `json.RawMessage`); text: `fmt.Fprintf(stdout, "REPLAY_GAP requested_after=%d oldest_available=%d\n", result.ReplayGap.GetRequestedAfterSequence(), result.ReplayGap.GetOldestAvailableSequence())`; then the shared `count++` / limit check, `continue`.
+2. New test in `clients/go/cmd/mxgw-go/main_test.go`: `TestRunStreamEventsPrintsReplayGap` — reuse the fake-gateway pattern (`TestRunPingPlainText`, `main_test.go:195`): a fake `StreamEvents` server implementation sends one sentinel `MxEvent` with `replay_gap{requested_after_sequence: 7, oldest_available_sequence: 42}` set, then one normal data event, then returns. Run `stream-events` in text mode and assert stdout contains the typed line `REPLAY_GAP requested_after=7 oldest_available=42` and does **not** contain `0 MX_EVENT_FAMILY_UNSPECIFIED`; run again with `-json` and assert a `"replayGap"` object with both sequences.
+3. Docs: the same `docs/CrossLanguageSmokeMatrix.md:33-39` per-CLI rendering row as CLI-35 (Go: typed `REPLAY_GAP` line / `replayGap` JSON row) — land the doc row once, covering both findings.
+
+**Verification.** From `clients/go`: `gofmt -l .` (clean), `go build ./...`, `go test ./...` (targeted first: `go test ./cmd/mxgw-go -run TestRunStreamEventsPrintsReplayGap`).
+
+---
+
+## CLI-37 — Status-array validation must branch on `category`, per the proto's own rule   `Medium` · `P1`
+
+**Finding.** The wire contract (`src/ZB.MOM.WW.MxGateway.Contracts/Protos/mxaccess_gateway.proto:982-992`) states that `MxStatusProxy.success` "is NOT a boolean … carried verbatim for diagnostics; the authoritative success/failure of the operation is `category` (MX_STATUS_CATEGORY_OK marks success) … Clients should branch on `category`, not on a specific `success` value." Four clients branch only on `success`: Rust `clients/rust/src/error.rs:326` (`status.success == 0` — written fresh for CLI-03), Go `clients/go/mxgateway/status.go:4-6` (`GetSuccess() != 0`), Java `clients/java/zb-mom-ww-mxgateway-client/src/main/java/com/zb/mom/ww/mxgateway/client/MxStatuses.java:26`, Python `clients/python/src/zb_mom_ww_mxgateway/errors.py:141`. .NET requires **both** (`clients/dotnet/ZB.MOM.WW.MxGateway.Client/MxStatusProxyExtensions.cs:11-17`: `Success != 0 && Category is MxStatusCategory.Ok`). A reply with `success = 1, category = COMMUNICATION_ERROR` passes four clients and throws in .NET; `success = 0, category = OK` does the reverse.
+
+**Impact.** The wire contract and four of five implementations disagree; identical gateway replies produce opposite outcomes per language. This is the same validation function CLI-38 touches — the two must land together as one conformance pass.
+
+**Design — canonical behavior (this is a conformance exercise, not five fixes).** Per the proto comment — which pre-dates every implementation and records the worker's mapping intent (`success` is the raw 16-bit COM value carried verbatim; `category` is the worker's normalized verdict) — the canonical per-entry rule is:
+
+> **An `MxStatusProxy` entry represents failure iff `category != MX_STATUS_CATEGORY_OK`. The `success` field is diagnostics only and never participates in the verdict.**
+
+Edge decisions, uniform across all five: an absent entry (Go `nil`, Java `null`) remains success (nothing to report — preserves existing nil-tolerance); a **present** entry with `MX_STATUS_CATEGORY_UNSPECIFIED` (0) is a **failure** (an unmapped category is not proven OK, and the worker always maps one — an unspecified category on the wire is itself anomalous). Rejected alternatives: (a) declare the proto comment wrong and standardize on `success` — rejected: the comment is the contract, `success` is explicitly documented as a raw non-boolean diagnostic, and rewriting the contract to match four accidental implementations inverts the authority relationship; (b) require both, as .NET does today — rejected: makes `success = 0, category = OK` fail, directly contradicting "the authoritative … is `category`".
+
+**Implementation** (one cross-client conformance commit, co-landed with CLI-38 since the same `EnsureMxAccessSuccess`-family functions change):
+1. **Shared fixtures first** — add to `clients/proto/fixtures/behavior/command-replies/`: `write.status-category-error-success-set.reply.json` (protocol OK, one status with `success: 1, category: MX_STATUS_CATEGORY_COMMUNICATION_ERROR` → expected **failure**) and `write.status-category-ok-success-zero.reply.json` (`success: 0, category: MX_STATUS_CATEGORY_OK` → expected **success**); register both in `clients/proto/fixtures/behavior/manifest.json` and describe them in `docs/ClientBehaviorFixtures.md`.
+2. **.NET** `clients/dotnet/ZB.MOM.WW.MxGateway.Client/MxStatusProxyExtensions.cs:11-17`: `IsSuccess` → `return status.Category is MxStatusCategory.Ok;` (drop the `Success != 0` conjunct); update the XML doc (line 8) accordingly.
+3. **Go** `clients/go/mxgateway/status.go:4-6`: `StatusSucceeded` → `return status == nil || status.GetCategory() == pb.MxStatusCategory_MX_STATUS_CATEGORY_OK` (import the generated enum); rewrite the function comment.
+4. **Java** `MxStatuses.java:25-27`: `succeeded` → `return status == null || status.getCategory() == MxStatusCategory.MX_STATUS_CATEGORY_OK;`; fix the class-level Javadoc (`:10-12`) and the `success()` accessor Javadoc (`:46-47`), which currently document the "non-zero success" convention.
+5. **Python** `clients/python/src/zb_mom_ww_mxgateway/errors.py:140-141`: `if mx_status.success == 0:` → `if mx_status.category != pb.MX_STATUS_CATEGORY_OK:`.
+6. **Rust** `clients/rust/src/error.rs:326`: `status.success == 0` → `status.category != MxStatusCategory::Ok as i32` (prost stores enums as `i32`); update the doc comment at `:313-314` ("A MXSTATUS_PROXY entry is treated as a failure when its `success` member is `0`") to the category rule; adjust the existing tests at `error.rs:410-452` that construct failing statuses via `success`.
+7. Per-client tests: wire the two new fixtures into each fixture-driven suite (Go fixture event/reply tests, `clients/python/tests/`, `MxGatewayClientSessionTests.cs`, `MxGatewayClientSessionTests.java`, `clients/rust/tests/client_behavior.rs`), asserting failure/success exactly per the canonical rule.
+8. Docs same-commit: `docs/ClientLibrariesDesign.md` Typed Command Parity section — extend the reply-validation sentence to state the per-item rule ("per-item `MxStatusProxy` failure iff `category != OK`; `success` is diagnostic only"); `docs/ClientBehaviorFixtures.md` (step 1).
+
+**Verification.** All five, per CLAUDE.md: `dotnet build clients/dotnet/ZB.MOM.WW.MxGateway.Client.slnx` + `dotnet test` (client tests); `gofmt`/`go build ./...`/`go test ./...` from `clients/go`; `JAVA_HOME=/opt/homebrew/opt/openjdk@17 gradle test` from `clients/java` (**revert the regenerated `MxaccessGateway.java` churn afterward — no `.proto` changed**); `python -m pytest` from `clients/python`; `cargo fmt` + `cargo check --workspace` + `cargo test --workspace` + `cargo clippy --all-targets -- -D warnings` from `clients/rust`.
+
+---
+
+## CLI-38 — Align .NET/Go/Java on `hresult < 0` (lands prior CLI-08; cures the doc drift)   `Medium` · `P1`
+
+**Finding.** `docs/ClientLibrariesDesign.md:118-119` (the Typed Command Parity section added with CLI-04) claims every client "runs the same MXAccess-level reply validation (HRESULT `< 0` + per-item `MxStatusProxy`)". Actual: .NET `clients/dotnet/ZB.MOM.WW.MxGateway.Client/MxCommandReplyExtensions.cs:34` (`reply.HasHresult && reply.Hresult != 0`), Go `clients/go/mxgateway/errors.go:187` (`reply.Hresult != nil && reply.GetHresult() != 0`), Java `clients/java/zb-mom-ww-mxgateway-client/.../MxGatewayErrors.java:50` (`reply.hasHresult() && reply.getHresult() != 0`) all use `!= 0`. Only Python (`errors.py:133`) and Rust (`error.rs:325`) implement the documented `< 0`.
+
+**Impact.** A parity-preserving reply carrying a positive COM success code (`S_FALSE = 1`) errors in three languages and succeeds in two — and every CLI-04 helper (WriteSecured, AuthenticateUser, …) inherits the divergence. The load-bearing design doc describes behavior three clients don't have.
+
+**Design.** **This finding is the vehicle that finally closes prior-cycle CLI-08** — the fix was fully designed in `archreview/remediation/50-clients.md` (CLI-08 section) two cycles ago and never landed; landing it now is strictly better than the alternative (correcting the doc to describe the divergence), because it makes the already-written doc true, closes CLI-08 and CLI-38 simultaneously, and completes the COM-correct semantics Rust/Python already have. Canonical rule, per COM semantics and the existing doc: **HRESULT failure iff `hresult` is present and `< 0`** (negative = failure; positive success codes like `S_FALSE` pass). Rejected alternative: doc-only correction — leaves a real cross-client behavioral divergence in the shipped typed surface and keeps CLI-08 open for a third cycle. Co-land with CLI-37: same functions, one conformance commit.
+
+**Implementation.**
+1. **.NET** `MxCommandReplyExtensions.cs:34`: `bool hResultFailure = reply.HasHresult && reply.Hresult < 0;`
+2. **Go** `errors.go:187`: `if reply.Hresult != nil && reply.GetHresult() < 0 {`
+3. **Java** `MxGatewayErrors.java:50`: `if (reply.hasHresult() && reply.getHresult() < 0) {`
+4. **Shared fixtures**: add `clients/proto/fixtures/behavior/command-replies/write.hresult-s-false.reply.json` (protocol OK, `hresult: 1` → expected **success**) and `write.hresult-e-fail.reply.json` (`hresult: -2147467259` /0x80004005/ → expected **failure**) to the behavior manifest; wire into all five suites (Python/Rust already assert `< 0` in unit tests — the fixture locks it cross-client).
+5. Docs/tracking same-commit: no change needed to `docs/ClientLibrariesDesign.md:118-119` — the change makes it true. Update the remediation tracking (`archreview/remediation/00-tracking.md` CLI-08 row → Done, noting it landed via CLI-38) and note the corrected semantics in the .NET/Go/Java README error sections (per the prior CLI-08 design).
+
+**Verification.** Per language: `dotnet build clients/dotnet/ZB.MOM.WW.MxGateway.Client.slnx` + client tests (new case: `hresult = 1` passes, negative fails); `go test ./...` from `clients/go`; `JAVA_HOME=/opt/homebrew/opt/openjdk@17 gradle test` from `clients/java` (revert generated churn — no `.proto` changed). Python/Rust suites run unchanged as regression (`python -m pytest`; `cargo test --workspace`).
+
+---
+
+## CLI-39 — Bump versions off the already-published 0.1.2; guard the publish pipeline   `Medium` · `P1`
+
+**Finding.** All four non-Java clients pin the *already-published* 0.1.2: `clients/rust/Cargo.toml:3` (`[package]`) and `:28` (`[workspace.package]`, inherited by `crates/mxgw-cli` via `version.workspace = true`), `clients/python/pyproject.toml:9` + `clients/python/src/zb_mom_ww_mxgateway/version.py:3`, `clients/go/mxgateway/version.go:7`, `clients/dotnet/ZB.MOM.WW.MxGateway.Client/ZB.MOM.WW.MxGateway.Client.csproj:22`. Java is at 0.2.0 (`clients/java/build.gradle:16`). Since the 0.1.2 publish, the public API changed incompatibly: Rust's `EventStream` item type became `Result<EventItem, Error>` and `Error` gained a variant; Python's `stream_events` yields `MxEvent | ReplayGap`; Go's `EventResult` gained `ReplayGap` and the `Events()` overflow contract changed to a terminal `ErrSlowConsumer`. The next `pack-clients.ps1` run either collides with the existing registry artifact or ships a different API under an identical version string.
+
+**Impact.** Registry rejection at best; at worst an identical version labeling two different APIs for anyone who cached 0.1.2. Release hygiene for the whole client family.
+
+**Design.** Bump **all five clients to 0.2.0** before the next publish. Rationale: the stream-surface changes are breaking, so under 0.x semver conventions (cargo treats 0.x minor as breaking) the right bump is minor, not patch — and 0.2.0 aligns every client on one number (Java is already there), simplifying the cross-language matrix and the e2e runner. Landing order: this is the **last** entry in the P0/P1 train — bump after CLI-35/36/37/38/45 so the published 0.2.0 carries the conformant behavior, not a third behavior set. Additionally, close the two publish-pipeline gaps this cycle re-confirmed: the tag-time Go version guard designed for CLI-21 but never implemented, and a registry-collision check in `scripts/pack-clients.ps1` (coordinate with the existing publish process: Gitea registry, credentials in `~/.zshenv`, Go module tags via `scripts/tag-go-module.ps1` with `clients/go/vX.Y.Z` prefixed tags). Rejected alternatives: (a) 0.1.3 patch bump — mislabels breaking changes and cargo consumers with `^0.1` would auto-upgrade into them; (b) per-language different bumps (0.2.0 Rust/Python, 0.1.3 Go/.NET) — legal but forfeits the alignment that makes the matrix and publish script auditable.
+
+**Implementation.**
+1. `clients/rust/Cargo.toml:3` and `:28`: `version = "0.2.0"` (both `[package]` and `[workspace.package]`; `crates/mxgw-cli` inherits; `CLIENT_VERSION` derives via `env!("CARGO_PKG_VERSION")` — no other Rust edit).
+2. `clients/python/pyproject.toml:9` and `clients/python/src/zb_mom_ww_mxgateway/version.py:3`: `0.2.0` in both. Add/extend a test asserting `version.__version__` equals the version parsed from `pyproject.toml` (closes the CLI-26 residual drift mode the review noted — `test_cli.py:213` currently only asserts self-consistency).
+3. `clients/go/mxgateway/version.go:7`: `ClientVersion = "0.2.0"`. The module tag `clients/go/v0.2.0` is created at publish time via `scripts/tag-go-module.ps1`.
+4. `clients/dotnet/ZB.MOM.WW.MxGateway.Client/ZB.MOM.WW.MxGateway.Client.csproj:22`: `<Version>0.2.0</Version>`.
+5. `clients/java/build.gradle:16`: stays `0.2.0` **only after verifying** (Gitea packages API) that Java 0.2.0 was not already published with the pre-remediation API; if it was, bump Java to 0.2.1 and record the exception in the packaging doc.
+6. `scripts/tag-go-module.ps1`: implement the CLI-21 guard that was designed but skipped — after semver validation, read `clients/go/mxgateway/version.go` and fail the tag when `ClientVersion` ≠ the tag version (strip the `v`).
+7. `scripts/pack-clients.ps1`: before each per-language publish step, query the Gitea package API for the target name+version and abort with a clear error when the artifact already exists (never force-overwrite). Reuse the verify-API calls the publish workflow already uses post-push.
+8. Docs same-commit: grep `0\.1\.2` across `docs/ClientPackaging.md` and the five client READMEs; update any stale version references; note the version-bump-before-publish rule in `docs/ClientPackaging.md`'s versioning section.
+
+**Verification.** Per-language builds prove the constants compile: `cargo check --workspace` + the existing `version.rs` test; `python -m pytest` (new pyproject-vs-`__version__` test); `go build ./...` + `go test ./...`; `dotnet build clients/dotnet/ZB.MOM.WW.MxGateway.Client.slnx`; `JAVA_HOME=/opt/homebrew/opt/openjdk@17 gradle test` only if `build.gradle` changes (revert generated churn). Script checks: `pwsh -NoProfile -Command` dry-run of `tag-go-module.ps1` with a mismatched version (must fail) and a matched one (must pass); `pack-clients.ps1` guard exercised against the existing 0.1.2 artifacts (must refuse) — actual publish happens only when the release train is cut.
+
+---
+
+## CLI-40 — Port the exact-secret credential scrub to Rust/Java/.NET   `Low` · `—`
+
+**Finding.** The credential-redaction seam is uniform in name only. Go scrubs the *exact secret values* from any surfaced error (`clients/go/mxgateway/errors.go:16-66` `secretRedactingError`/`redactSecrets`, applied at `session.go:745`/`:836` for WriteSecured payloads and the AuthenticateUser password); Python does the same (`session.py:573-592` `_invoke_redacted`). Rust scrubs only whitespace-delimited tokens shaped like API keys (`clients/rust/src/error.rs:363-375` `redact_credentials`: `mxgw_` prefix or `bearer`); Java likewise (`clients/java/zb-mom-ww-mxgateway-client/.../MxGatewaySecrets.java:44-57`); .NET has **no library-level scrub** (exception text embeds `status.DiagnosticText` verbatim via `MxStatusProxyExtensions.ToDiagnosticSummary`; only the CLI redacts, `MxGatewayCliSecretRedactor.cs`). Yet `docs/ClientLibrariesDesign.md:123-127` and the Rust helper docs (`session.rs:876-883`) claim credentials can never reach exception text. The per-client tests pass because the fakes never echo the credential.
+
+**Impact.** If the gateway or MXAccess echoes a credential back in `diagnostic_text` (which the client cannot prevent), Rust/Java/.NET surface it verbatim in exception messages — contradicting the documented guarantee. Low severity (requires a server-side echo), but the doc claims more than three clients deliver.
+
+**Design — canonical behavior.** Every credential-bearing helper (`AuthenticateUser` password, `WriteSecured`/`WriteSecured2` string payloads) must scrub the **exact secret values** it was called with from any error text it surfaces, replacing each occurrence with `<redacted>` (the marker all five already use). This is defense-in-depth on top of the by-construction guarantee (exceptions carry reply-derived text, never the request). Go and Python define the reference semantics: wrap/post-process the error at the credential-bearing call site; typed-error matching (`errors.Is`/`except`/`catch`) must keep working through the redaction. Rejected alternative: downgrade the doc claim to "requests are never embedded in errors" — weaker guarantee than two clients already deliver and than the docs have promised since CLI-04; porting the seam is bounded work.
+
+**Implementation.**
+1. **Rust**: extend `MxAccessError` (and the error path used by credential helpers) to carry `secrets: Vec<String>`; in its `Display` (which already funnels through `redact_credentials`, `error.rs:363`), first replace each exact secret occurrence with `<redacted>`, then apply the existing pattern scrub. Credential helpers in `clients/rust/src/session.rs` (`authenticate_user`, `write_secured`, `write_secured2`) attach the secrets when mapping errors. Keep the helper-doc claim at `session.rs:876-883` — it becomes true.
+2. **Java**: add `MxGatewaySecrets.redactExact(String message, String... secrets)` (exact-substring replacement, null/blank-tolerant) alongside the pattern scrub; route the credential commands in `MxGatewaySession.java` (`authenticateUser`, `writeSecured`, `writeSecured2`) through a private `invokeCommandRedacted(command, String... secrets)` that catches `MxGatewayException`, rebuilds the message via `redactExact`, and rethrows the **same exception type** (mirror Python's `_invoke_redacted`).
+3. **.NET**: introduce a library-level equivalent in `MxGatewaySession.cs`: the credential helpers (`AuthenticateUserAsync`, `WriteSecuredAsync`, `WriteSecured2Async`) wrap their invoke+validate in a catch that rethrows the same exception type with each exact secret replaced by `<redacted>` in `Message` (reuse the `<redacted>` marker; factor the replacement into a small internal `MxGatewaySecretRedaction` helper so the CLI's `MxGatewayCliSecretRedactor` can share it).
+4. **Shared fixture**: add `clients/proto/fixtures/behavior/command-replies/authenticate-user.echoed-credential.reply.json` — an MXAccess-failure reply whose `diagnostic_text` embeds the fixture credential string; every client's suite asserts the surfaced error message does **not** contain the credential and does contain `<redacted>`. Register in the manifest + `docs/ClientBehaviorFixtures.md`.
+5. Docs: none beyond the fixture doc — the design-doc claim becomes accurate.
+
+**Verification.** `cargo fmt`/`check`/`test`/`clippy -D warnings` from `clients/rust`; `JAVA_HOME=/opt/homebrew/opt/openjdk@17 gradle test` from `clients/java` (revert generated churn); `dotnet build clients/dotnet/ZB.MOM.WW.MxGateway.Client.slnx` + client tests; Go/Python suites re-run the new shared fixture as regression.
+
+---
+
+## CLI-41 — Uniform malformed-reply contract for the id/handle-returning helpers   `Low` · `—`
+
+**Finding.** A gateway reply that omits the typed payload yields five different behaviors. `AuthenticateUser`: Go falls back to `GetReturnValue().GetInt32Value()` → silent `0` when both are absent (`clients/go/mxgateway/session.go:807-816`); Python reads `reply.authenticate_user.user_id` → proto3 default `0`, no fallback, no error (`clients/python/src/zb_mom_ww_mxgateway/session.py:713`); Java falls back like Go → `0` (`MxGatewaySession.java:868-880`); .NET `reply.AuthenticateUser?.UserId ?? reply.ReturnValue.Int32Value` → `NullReferenceException` when both are absent (`clients/dotnet/.../MxGatewaySession.cs:1289`); Rust returns a typed `Error::MalformedReply` (`clients/rust/src/session.rs:1073-1089`) — but Rust's own `add_buffered_item_handle` (`:1057-1071`) *does* fall back to `return_value`, so Rust is inconsistent internally.
+
+**Impact.** Silent `0` from `AuthenticateUser` is the worst mode — callers feed it into `WriteSecured` as a real user id. The NRE is a crash on a malformed-but-received reply.
+
+**Design — canonical behavior.** For every helper that extracts a scalar from a reply (`AuthenticateUser`, `ArchestrAUserToId`, `AddBufferedItem`, and the existing `AddItem2`-style handle extractors):
+
+> **Prefer the typed payload; when it is absent, fall back to `return_value` only if `return_value` is actually present with the expected variant; when neither is present, raise/return a typed malformed-reply error. Never surface a proto3 default `0`, never NRE.**
+
+This is Rust's `add_buffered_item_handle` pattern (payload → present `return_value` → `MalformedReply`), generalized. The `return_value` fallback is kept because it is the documented compatibility path for replies from workers that populate only the legacy field; the typed error replaces only the "both absent" case. Rejected alternatives: (a) Rust's stricter payload-or-error (no fallback, current `authenticate_user_id`) — breaks the legacy-`return_value` compatibility the other four clients and Rust's own handle extractors honor; (b) documenting the divergence — a silent-`0` user id is a correctness hazard, not a doc gap.
+
+**Implementation.**
+1. **Go** `clients/go/mxgateway/session.go`: add a `MalformedReplyError{Op string, Detail string}` type to `errors.go` (Error/Unwrap in the house style). In `AuthenticateUser` (`:807`), `ArchestrAUserToId`, and `AddBufferedItem`: check the typed payload; else `if reply.ReturnValue != nil` use it; else return the typed error.
+2. **Python** `clients/python/src/zb_mom_ww_mxgateway/session.py` + `errors.py`: add `MalformedReplyError(MxGatewayError)`. In `authenticate_user` (`:713`), `archestra_user_to_id`, `add_buffered_item`: `if reply.HasField("authenticate_user")` → typed payload; `elif reply.HasField("return_value")` and the value oneof is `int32_value` → fallback; else raise.
+3. **Java** `MxGatewaySession.java:868-880` (+ `archestrAUserToId`, `addBufferedItem`): `if (reply.hasAuthenticateUser())` → payload; `else if (reply.hasReturnValue())` → fallback; else throw a new `MxGatewayMalformedReplyException extends MxGatewayException`.
+4. **.NET** `MxGatewaySession.cs:1289` (+ the archestra/buffered equivalents at `:1332`/`:935`): `reply.AuthenticateUser?.UserId ?? reply.ReturnValue?.Int32Value ?? throw new MxGatewayMalformedReplyException(...)` (new exception type deriving from `MxGatewayException`).
+5. **Rust** `clients/rust/src/session.rs:1073-1089`: add the `return_value` fallback to `authenticate_user_id` and `archestra_user_id`, matching `add_buffered_item_handle` — payload → `return_value` via `int32_reply_value` → `Error::MalformedReply`.
+6. **Shared fixtures**: `clients/proto/fixtures/behavior/command-replies/authenticate-user.missing-payload.reply.json` (protocol OK, no payload, no `return_value` → expected typed error) and `authenticate-user.return-value-only.reply.json` (`return_value.int32_value = 7`, no typed payload → expected `7`); manifest + `docs/ClientBehaviorFixtures.md` + all five suites.
+7. Docs: one sentence in `docs/ClientLibrariesDesign.md`'s Typed Command Parity section stating the payload → return_value → typed-error contract.
+
+**Verification.** All five per CLAUDE.md: dotnet slnx build + tests; `go test ./...`; `JAVA_HOME=/opt/homebrew/opt/openjdk@17 gradle test` (revert generated churn); `python -m pytest`; `cargo fmt`/`check`/`test`/`clippy -D warnings`.
+
+---
+
+## CLI-42 — Document the vendored Rust proto layout (CLI-02's missing doc half)   `Low` · `P1 (doc batch)`
+
+**Finding.** CLI-02's code fix landed (repo-path-first/vendored-fallback `build.rs`, `clients/rust/protos/*.proto` vendored and `include`d in `Cargo.toml:20`, `--no-verify` removed from packaging) but the documentation half was skipped: `clients/rust/README.md:21-25` still says `build.rs` reads only `../../src/ZB.MOM.WW.MxGateway.Contracts/Protos`, and `docs/ClientPackaging.md`'s Rust section (`:114-127`) never mentions `clients/rust/protos/` (repo-wide grep for "vendored" in `docs/*.md` → zero hits). The vendored copies are load-bearing published build inputs with a refresh obligation on every `.proto` change, currently enforced only by `scripts/check-codegen.ps1` Check 3 and a `build.rs` comment. CLAUDE.md's docs-in-same-commit rule was not met.
+
+**Impact.** Anyone editing the contracts protos has no prose telling them the Rust crate ships its own copies that must be refreshed; the packaging doc describes a crate layout that no longer matches what `cargo package` ships.
+
+**Design.** Pure documentation: describe the resolution order and the refresh rule in both places the review names. No code. Batched with the other P1 doc work.
+
+**Implementation.**
+1. `clients/rust/README.md:21-25`: rewrite the generation paragraph — `build.rs` resolves protos repo-path-first (`../../src/ZB.MOM.WW.MxGateway.Contracts/Protos`, keeping in-repo `.proto` edits live) and falls back to the vendored copies in `clients/rust/protos/` (shipped in the published crate via `Cargo.toml`'s `include`, making the tarball self-sufficient — this is what `cargo package` verification exercises). State the refresh rule: any change to the Contracts protos must copy the changed files into `clients/rust/protos/` in the same commit; `scripts/check-codegen.ps1` (Check 3) fails on byte drift.
+2. `docs/ClientPackaging.md` Rust section (after the `build.rs` sentence around `:116-118`): add the same vendored-layout paragraph plus a note that `cargo package`/`cargo publish` run **with** verification (no `--no-verify`) precisely because the vendored protos make the standalone build possible.
+
+**Verification.** `grep -rn "protos/" clients/rust/README.md docs/ClientPackaging.md` shows the vendored dir documented in both; `grep -i vendored docs/ClientPackaging.md clients/rust/README.md` non-empty. No build required (docs only).
+
+---
+
+## CLI-43 — Java style guide still prescribes "Java 21 preferred"   `Low` · `—`
+
+**Finding.** `docs/style-guides/JavaStyleGuide.md:8` — "Target the Java version defined by the client build, with Java 21 preferred." CLAUDE.md declares `docs/style-guides/` authoritative, and this line contradicts the shipped `toolchain 17` / `options.release = 17` build (`clients/java/build.gradle`) and the CLI-12-corrected docs. (Historical plan docs also say 21 but are archival — leave them.)
+
+**Impact.** An authoritative guide instructing contributors toward the wrong language level for the Ignition 8.3 target.
+
+**Design.** One-line doc fix, mirroring CLI-12's wording. No code.
+
+**Implementation.**
+1. `docs/style-guides/JavaStyleGuide.md:8`: replace with "Target Java 17 (the Ignition 8.3 baseline; the client build enforces `options.release = 17` with a Gradle toolchain 17). Code must compile and run on 17; newer JDKs may host the build."
+
+**Verification.** `grep -rn "Java 21" docs/style-guides/` → empty. No build (docs only).
+
+---
+
+## CLI-44 — Go event goroutine can mislabel a genuine terminal error as `ErrSlowConsumer`   `Low` · `—`
+
+**Finding.** When `stream.Recv()` returns a real terminal error while the 16 data slots are full, the Recv-error path (`clients/go/mxgateway/session.go:1051-1057`) reuses `sendEventResult`, whose overflow branch (`:1082-1101`) fires first: it substitutes `ErrSlowConsumer` for the caller's `GatewayError{Err: err}`. The consumer gets a loud terminal error (good) with the wrong root cause — the real gRPC status is lost.
+
+**Impact.** Misleading diagnostics on a narrow race (full buffer + genuine stream failure), not silent loss — hence Low.
+
+**Design.** Terminal sends must bypass the overflow substitution and use the reserved slot directly with the caller's error. The reserved slot (`eventBufferReservedSlots`, `session.go:47-56`) exists exactly to guarantee one terminal result is always deliverable; the sole-producer invariant (CLI-01's design) means at most one terminal send ever races for it. Rejected alternative: enlarging the reserve to two slots so both errors can be delivered — the consumer only ever acts on the first terminal error; delivering two contradicting terminal results complicates the contract for no diagnostic gain.
+
+**Implementation.**
+1. `clients/go/mxgateway/session.go`: add a small `sendTerminalEventResult(results chan<- EventResult, result EventResult)` doing a non-blocking `select { case results <- result: default: }` (comment: the reserved slot guarantees the send unless a terminal result was already enqueued). Use it in the Recv-error path (`:1051-1057`) instead of `sendEventResult` (the stream has already ended — no `cancel()` needed beyond the deferred cleanup; keep the existing `cancel` call if one is required for stream teardown symmetry). The overflow branch in `sendEventResult` is unchanged.
+2. New test in `clients/go/mxgateway/client_session_test.go`: `TestEventsFullBufferTerminalErrorKeepsRootCause` — fill all 16 data slots without draining (reuse the CLI-01 slow-consumer test scaffolding at `:150-185`), then make the fake stream's `Recv` return a non-EOF gRPC error; drain and assert the final `EventResult.Err` unwraps to the gRPC status error and `errors.Is(err, ErrSlowConsumer)` is **false**.
+3. Docs: adjust the `Events`/`EventsAfter` doc comments if they state that a full buffer always yields `ErrSlowConsumer` (the contract becomes: overflow yields `ErrSlowConsumer`; a genuine stream error is reported as itself even under overflow).
+
+**Verification.** From `clients/go`: `gofmt -l .`, `go build ./...`, `go test ./...` (targeted: `go test ./mxgateway -run TestEventsFullBufferTerminalErrorKeepsRootCause`).
+
+---
+
+## CLI-45 — Standardize CLI credential env-var name; fail fast on missing/empty passwords   `Low` · `P1`
+
+**Finding.** The new `authenticate-user` CLI credential flags diverge per language. Go (`clients/go/cmd/mxgw-go/main.go:441-457`): `-password`/`-password-env`, default env `MXGATEWAY_VERIFY_PASSWORD`, but an unresolved (empty) password is **silently sent to the wire**. Java (`clients/java/zb-mom-ww-mxgateway-cli/.../MxGatewayCli.java:1136-1160`): same flag/env names, and an unset env falls back to `resolvedPassword = ""` — also sent as an empty credential. Rust (`clients/rust/crates/mxgw-cli/src/main.rs:136-155`): same names, missing → `Error::InvalidArgument` (conformant). Python (`commands.py:832-847` `_resolve_password`): no default env name — `--password-env` must be passed explicitly; missing → `click.UsageError`. .NET (`MxGatewayClientCli.cs:349-388`): different flag names (`--verify-user-password`/`--verify-user-password-env`) and a different default env (`MXGATEWAY_VERIFY_USER_PASSWORD`); missing → `ArgumentException`. Subcommand coverage also diverges (.NET exposes all nine new commands; Rust adds unregister + the two credential commands; Go/Python/Java add only `write-secured` + `authenticate-user`).
+
+**Impact.** The same operator workflow (`export MXGATEWAY_VERIFY_PASSWORD=… && <cli> authenticate-user …`) works in three CLIs, needs an explicit flag in Python, and needs a different variable in .NET — and Go/Java silently authenticate with an empty password, turning a misconfigured env into a real (failing or, worse, succeeding) MXAccess authentication attempt.
+
+**Design — canonical behavior.** One CLI credential contract for all five:
+
+> Flags `--password` (explicit value; discouraged — stays out of shell history via the env path) and `--password-env` (name of the environment variable; **default `MXGATEWAY_VERIFY_PASSWORD`**). Resolution order: flag, then env. When the resolved credential is **missing or empty**, the CLI fails fast with a usage error naming the flag and the env var — it never sends an empty password to the wire and never echoes the value.
+
+`MXGATEWAY_VERIFY_PASSWORD` wins as the canonical name because three of five CLIs and their docs already use it — changing .NET (one CLI, with aliases) is the smallest blast radius; rejected alternative: standardizing on .NET's `MXGATEWAY_VERIFY_USER_PASSWORD` would churn three CLIs plus the smoke matrix for no benefit. The fail-fast rule is CLI argument validation, not a parity violation: the *library* still transmits whatever credential it is given (MXAccess parity preserved); only the operator tool refuses to fabricate an empty one — matching what Rust/Python/.NET already do. Subcommand-coverage leveling is out of scope here (larger feature work); the deltas get documented instead.
+
+**Implementation.**
+1. **Go** `clients/go/cmd/mxgw-go/main.go` (`runAuthenticateUser`, after the resolution block at `:454-457`): `if resolvedPassword == "" { return fmt.Errorf("a password is required via -password or the %s environment variable", *passwordEnv) }` — before dialing.
+2. **Java** `MxGatewayCli.java` `AuthenticateUserCommand.call()` (`:1151-1159`): replace the `resolvedPassword = ""` fallback with a picocli `ParameterException` (via `common.spec.commandLine()`) when the resolved value is null or blank, message naming `--password` and the `--password-env` variable.
+3. **Python** `clients/python/src/zb_mom_ww_mxgateway_cli/commands.py`: give the `--password-env` click option the default `"MXGATEWAY_VERIFY_PASSWORD"` (both `authenticate-user` and any secured-write command that shares the option); `_resolve_password` keeps its `click.UsageError` (already conformant once the default exists).
+4. **.NET** `MxGatewayClientCli.cs` (`TryResolveVerifyUserPassword`/`ResolveVerifyUserPassword`, `:349-388`): accept `--password` and `--password-env` as the primary names with default env `MXGATEWAY_VERIFY_PASSWORD`; keep `--verify-user-password`, `--verify-user-password-env`, and the `MXGATEWAY_VERIFY_USER_PASSWORD` env as deprecated aliases/fallbacks for one release (resolution order: `--password`, `--verify-user-password`, env named by `--password-env` (default `MXGATEWAY_VERIFY_PASSWORD`), legacy `MXGATEWAY_VERIFY_USER_PASSWORD`). The existing `ArgumentException` fail-fast stays; ensure the empty-string case (flag or env set but empty) also fails.
+5. **Rust** `mxgw-cli/src/main.rs:143-152`: verify the empty-string env case — if `std::env::var` returns an empty string it must be treated as missing (add the check if absent). Otherwise already conformant.
+6. Tests (new, per CLI): Go `TestRunAuthenticateUserRejectsEmptyPassword` in `main_test.go` (env unset → error mentioning `MXGATEWAY_VERIFY_PASSWORD`, no RPC attempted — no fake server needed); Java a picocli-level test invoking `authenticate-user` without credential and asserting the usage error; Python a `test_cli.py` case asserting the default env name is honored (monkeypatch `os.environ`) and that missing → `UsageError`; .NET CLI tests asserting the new names, the alias fallback, and the empty-value failure; Rust a case for empty-string env → `InvalidArgument`.
+7. Docs same-commit: `docs/CrossLanguageSmokeMatrix.md` — document the canonical flag/env contract in the smoke-command section and add a per-CLI subcommand-coverage table noting the current deltas (.NET: all nine; Rust: unregister + credential pair; Go/Python/Java: `write-secured` + `authenticate-user`) so the divergence is at least specified until a leveling pass; client READMEs' `authenticate-user` sections updated where they name the env var (.NET README gains the rename + deprecation note).
+
+**Verification.** Go: `gofmt -l .`, `go build ./...`, `go test ./...`. Java: `JAVA_HOME=/opt/homebrew/opt/openjdk@17 gradle test` from `clients/java` (**revert the `MxaccessGateway.java` regen churn — no `.proto` changed**). Python: `python -m pytest` from `clients/python`. .NET: `dotnet build clients/dotnet/ZB.MOM.WW.MxGateway.Client.slnx` + client/CLI tests. Rust: `cargo fmt`, `cargo check --workspace`, `cargo test --workspace`, `cargo clippy --all-targets -- -D warnings` from `clients/rust`.

archreview/2026-07-12/remediation/60-testing-docs-gaps.md

@@ -0,0 +1,154 @@
+# Testing, Documentation & Underdeveloped Areas — Remediation Design & Implementation (2026-07-12 cycle)
+
+Source review: [60-testing-docs-gaps.md](../60-testing-docs-gaps.md) · Baseline: `main` @ `4f5371f` · Generated: 2026-07-13
+
+This cycle's new findings (TST-25..29) are all fallout or residue of the prior remediation: the Windows CI tier was removed after act_runner proved broken (TST-25), the docs describing that tier were not updated in the same commit (TST-26), a flag SEC-25 made live is still documented as dead (TST-27), a handshake field is asserted only on the side CI cannot run (TST-28), and the root-artifact triage from TST-14 needs its retirement decision refreshed (TST-29). Every citation below was re-verified against the working tree on 2026-07-13.
+
+Prior-cycle open findings (TST-05..24 where still open) are tracked in the prior design doc, `archreview/remediation/60-testing-docs-gaps.md`; this doc covers only the new findings. Note TST-25 is the unlock for two of those: a working Windows tier is where the scheduled live-MXAccess smoke (old TST-05) and CI-run client wire tests (old TST-24) land.
+
+## Finding register
+
+| ID | Sev | Tier | Eff | Dep | Status | Title |
+|----|-----|------|-----|-----|--------|-------|
+| TST-25 | High | P1 | M | — (unlocks TST-05, TST-24) | Not started | Windows/x86 test tier has zero automation — restore via SSH-driven windev CI job |
+| TST-26 | Medium | P1 (folded into TST-25) | S | TST-25 | Not started | docs/GatewayTesting.md, check-codegen.ps1, and ci.yml comments describe removed CI jobs |
+| TST-27 | Medium | P1 (doc batch) | S | — | Not started | `ShowTagValues` config row still says "Reserved" after SEC-25 made the flag live |
+| TST-28 | Low | P2 | S | relates IPC-02 | Not started | Gateway-side `max_frame_bytes` handshake field untested in the CI-run suite |
+| TST-29 | Low | P2 | S | — | Not started | Retire `oldtasks.md` after folding the Phase-5 governance record into DesignDecisions.md; delete root docs-review artifacts |
+
+---
+
+## TST-25 — Windows/x86 test tier has zero automation   `High` · `P1` · Effort M
+
+**Finding.** The `windows` and `live-mxaccess` CI jobs were removed in `abb0930` because Gitea act_runner v0.6.1 host-mode is broken on Windows and a `runs-on`-gated job with no matching runner wedges every run in "queued" (removal note: `.gitea/workflows/ci.yml:123-131`). Consequence: the x86/net48 Worker build, all of `src/ZB.MOM.WW.MxGateway.Worker.Tests` — including the tests for remediation-added machinery that exists nowhere else (`WorkerFrameWritePriority` ordering at `Worker.Tests/Ipc/WorkerFrameProtocolTests.cs:353-395`, worker-side `max_frame_bytes` adoption, WnWrap/subtag alarm consumers, `StaRuntime` changes) — and the 8-fact live suite (`src/ZB.MOM.WW.MxGateway.IntegrationTests/WorkerLiveMxAccessSmokeTests.cs`) run only when someone remembers the manual windev worktree process. The portable job's `check-codegen.ps1` Generated-diff check substitutes for the net48 CS0246 guard; nothing substitutes for actually compiling and running the worker tests.
+
+**Impact.** A whole tier of *unit* tests — not just live smoke — is unguarded. A worker-side regression (frame priority inversion, frame-max adoption break, alarm-consumer conversion bug, STA loop change) ships green through CI and is caught, if at all, by hand on windev or in production. This regressed the prior cycle's TST-03/TST-05 posture and blocks old TST-24.
+
+**Design.** Three candidates, weighed against the operational facts (Gitea Actions runs on a co-located `gitea-runner` container on docker host `10.100.0.35`, job containers on network `traefik` resolving `gitea:3000`; windev is Windows 10 at `10.100.0.48`, ssh alias `windev`, passwordless ssh from the Mac, MXAccess installed, and a proven isolated-worktree flow already exists in `C:\build` driven by remote PowerShell via base64 `-EncodedCommand`):
+
+- **(a) Retry a Windows act_runner in host mode with the right labels — rejected.** The failure is not a label problem: act_runner v0.6.1's hostexecutor writes each step's script to a path it then cannot resolve (independent of shell) and cannot stage JS actions, so no label configuration fixes it; Windows containers are impractical for a net48/x86/MXAccess build. And the structural hazard remains: any `runs-on` gate with no live runner sticks queued forever, so a flaky/offline Windows runner would block *every* run. Revisit only if a fixed act_runner release ships — it would then replace the SSH job with a native one.
+- **(b) SSH-driven job — recommended.** A Linux CI job (same `ubuntu-latest` runner class as `portable`, so it always schedules) ssh's to windev, fast-forwards an isolated CI clone under `C:\build` to the exact SHA under test, runs the x86 build + `Worker.Tests` there, and propagates the remote exit code through ssh → step → job. This reuses the already-proven windev worktree + `-EncodedCommand` recipe, requires no new runner software on Windows, and cannot wedge the queue (the Linux runner always exists; an unreachable windev fails the job red — loud, not stuck).
+- **(c) Trigger split — adopted as part of (b).** Worker tests have nontrivial wall time on windev, and the live suite needs provider state and must never gate a push. Split: **per-push/PR** job runs `dotnet build Worker.csproj -p:Platform=x86` + `dotnet test Worker.Tests -p:Platform=x86` (unit suite — expected minutes; if measured wall time exceeds ~10 min, demote the test step to nightly and keep the per-push x86 build, which alone catches the net48/CS0246/x86 compile class); the **nightly** scheduled job (the existing `cron: '0 6 * * *'`, currently pointless) re-runs the x86 build + full Worker.Tests on `main`, then runs the live-MXAccess smoke with `MXGATEWAY_RUN_LIVE_MXACCESS_TESTS=1`, and optionally a full `src/ZB.MOM.WW.MxGateway.slnx` build (catches Windows-only analyzer diagnostics like xUnit1030 that never surface on macOS/Linux).
+
+Design details for (b):
+
+- **Isolation.** A dedicated CI clone `C:\build\mxaccessgw-ci` — never the Desktop checkout (dirty feature branch, per the established windev rule). The job's inline bootstrap (base64 `-EncodedCommand`) does `git fetch origin && git checkout --force <sha>` in that clone, then invokes the *checked-out* `scripts/ci/windev-worker-ci.ps1 -Sha <sha> -Mode <build|test|live>` so the CI logic itself is versioned with the code under test; only the tiny bootstrap lives in ci.yml.
+- **Serialization.** One clone, potentially overlapping runs: `windev-worker-ci.ps1` acquires a lock (`New-Item C:\build\mxaccessgw-ci.lock -ItemType Directory`, retry loop with ~20 min timeout, always released in `finally`) so concurrent pushes queue instead of stomping the worktree. Add a workflow-level `concurrency` group too if the deployed Gitea version honors it — the lock is the guarantee, `concurrency` the nicety.
+- **Secrets.** Generate a dedicated ed25519 keypair for CI (do **not** reuse the operator's personal windev key). Private key → Gitea Actions repo secret `WINDEV_SSH_KEY`; pinned host key → secret `WINDEV_SSH_KNOWN_HOSTS` (or a committed `scripts/ci/windev.known_hosts` — host keys are public data; committing is acceptable and greppable). The job writes the key to `~/.ssh/id_ed25519` mode 0600 and connects with `-o BatchMode=yes -o StrictHostKeyChecking=yes -o UserKnownHostsFile=...`. On windev, append the pub key to `authorized_keys` for the CI account; optionally prefix with a forced `command=` restricting it to the bootstrap script. Per CLAUDE.md, the key must never be echoed into logs.
+- **Network precheck.** The job container is on the `traefik` docker network; confirm it has egress to `10.100.0.48:22` (a one-off `nc -z -w5 10.100.0.48 22` step during bring-up; docker bridge networks normally route to the LAN, but the network was created for gitea resolution, not egress, so verify rather than assume).
+- **Failure visibility.** Per-push runs are first-class CI jobs — a red windev run is a red commit/PR check exactly like `portable`/`java`, and an unreachable windev **fails** (never skips) so a down Windows tier is loud. Nightly runs surface on the repo's Actions page against `main`; because nobody watches that page, add a final `if: failure()` step to the nightly job that opens (or comments on) a Gitea issue via the API using the built-in actions token, so a red nightly produces a notification artifact. Document in `docs/GatewayTesting.md` that a red `windows-x86` job means the Windows tier — not the change — may be down, and how to check (ssh windev, read `C:\build\mxaccessgw-ci\ci.log`).
+
+**Implementation.**
+
+1. **Windev prep (one-time, manual):** create `C:\build\mxaccessgw-ci` as a fresh clone of the Gitea origin (read-only deploy token or the existing credential store); verify `dotnet build src/ZB.MOM.WW.MxGateway.Worker/ZB.MOM.WW.MxGateway.Worker.csproj -p:Platform=x86` and `dotnet test src/ZB.MOM.WW.MxGateway.Worker.Tests/ZB.MOM.WW.MxGateway.Worker.Tests.csproj -p:Platform=x86` succeed there by hand; record wall time (drives the per-push vs nightly split for the test step). Generate the CI keypair, install the pub key on windev, store `WINDEV_SSH_KEY` (+ known-hosts) in Gitea repo secrets.
+2. **New `scripts/ci/windev-worker-ci.ps1`** (runs *on windev*, invoked at the tested SHA): parameters `-Sha`, `-Mode build|test|live`; acquires/releases the lock dir; `build` = x86 Worker build; `test` = build + `dotnet test Worker.Tests -p:Platform=x86`; `live` = `test` + `MXGATEWAY_RUN_LIVE_MXACCESS_TESTS=1 dotnet test src/ZB.MOM.WW.MxGateway.IntegrationTests/... --filter FullyQualifiedName~WorkerLiveMxAccessSmokeTests` (+ optional full-slnx build); exits nonzero on any failure.
+3. **New `scripts/ci/run-windev-ci.sh`** (runs *in the Linux job*): writes the key/known-hosts from env, builds the base64 `-EncodedCommand` bootstrap (fetch + checkout `$CI_SHA` in `C:\build\mxaccessgw-ci`, then `powershell -File scripts\ci\windev-worker-ci.ps1 -Sha $CI_SHA -Mode $1`), runs `ssh -o BatchMode=yes ci@10.100.0.48 ...`, exits with the ssh exit code. Do not wrap the remote git calls in `$ErrorActionPreference='Stop'` (git writes progress to stderr — known windev gotcha).
+4. **ci.yml — add the per-push job** (same commit as steps 5–6):
+
+   ```yaml
+   windows-x86:
+     # x86/net48 Worker build + Worker.Tests, executed on windev (10.100.0.48) over SSH.
+     # Runs on a Linux runner so it always schedules (act_runner host-mode on Windows is
+     # broken; a runs-on gate with no runner wedges the queue — see docs/GatewayTesting.md).
+     runs-on: ubuntu-latest
+     steps:
+       - uses: actions/checkout@v4
+       - name: Worker x86 build + tests on windev
+         env:
+           WINDEV_SSH_KEY: ${{ secrets.WINDEV_SSH_KEY }}
+           WINDEV_SSH_KNOWN_HOSTS: ${{ secrets.WINDEV_SSH_KNOWN_HOSTS }}
+           CI_SHA: ${{ github.sha }}
+         run: ./scripts/ci/run-windev-ci.sh test   # demote to `build` if measured wall time > ~10 min
+   ```
+
+5. **ci.yml — add the nightly job** gated `if: github.event_name == 'schedule'` (safe: the gate is an `if`, not an unsatisfiable `runs-on`, so push runs mark it skipped, not queued), `runs-on: ubuntu-latest`, calling `run-windev-ci.sh live`; final `if: failure()` step posts a Gitea issue/comment via the API token. Fix the cron comment at `ci.yml:12-14` to describe what the schedule now actually runs.
+6. **ci.yml — replace the removal note** at `:123-131` with a short pointer: host-mode act_runner remains broken; the Windows tier runs via the SSH-driven `windows-x86`/nightly jobs; restore native jobs only with a proven runner.
+7. **TST-26 doc edits in the same commit** — see TST-26 below.
+8. After the first green nightly, mark old **TST-05** addressed (live smoke is scheduled again) and unblock old **TST-24** (client wire tests can join CI).
+
+**Verification.**
+- Push a branch: `portable`, `java`, and `windows-x86` all green; confirm on windev that `C:\build\mxaccessgw-ci` sits at the pushed SHA.
+- **Deliberate red:** on a scratch branch, add a failing `[Fact]` to `Worker.Tests` (or an intentional x86-only compile error), push, confirm `windows-x86` turns red and the run fails; revert. This proves exit-code propagation end-to-end, the whole point of the SSH design.
+- **Unreachable-host red:** temporarily point the script at a bogus port (or stop sshd), confirm the job fails fast with a clear message rather than hanging or passing.
+- **Concurrency:** push two branches back-to-back; confirm the second run's remote step waits on the lock and both complete.
+- **Nightly:** temporarily set the cron a few minutes ahead (or trigger the schedule path manually), confirm the live job runs with `MXGATEWAY_RUN_LIVE_MXACCESS_TESTS=1` and, with a forced failure, that the on-failure issue step fires.
+- Confirm no secret material appears in the job logs.
+
+---
+
+## TST-26 — Docs describe CI jobs that no longer exist   `Medium` · `P1 (folded into TST-25)` · Effort S
+
+**Finding.** `docs/GatewayTesting.md:426-431` still documents the **`windows`** job ("the only host that builds the x86/net48 Worker… the definitive guard for the 'regenerate and commit `Generated/`' rule") and the **`live-mxaccess`** scheduled job — both deleted from `.gitea/workflows/ci.yml` in `abb0930`, which touched only the workflow file (a violation of the repo's docs-change-in-same-commit rule). Same stale claim at `scripts/check-codegen.ps1:17` ("guarded by the Windows CI job, not here") and in the `ci.yml:12-14` cron comment ("Nightly live-MXAccess smoke (windev)") though the schedule currently re-runs only `portable`+`java`.
+
+**Impact.** Actively misleading in the worst direction: a reader concludes the worker build is CI-guarded when it is not, and the "definitive guard" claim is doubly wrong — the surviving guard for the Generated-commit rule is the portable job's `check-codegen.ps1` diff check, not a Windows compile.
+
+**Design.** No standalone commit: these edits land **in the same commit as the TST-25 ci.yml change** (that is the same-commit rule the removal violated). Content rules: describe the jobs that exist (`portable`, `java`, `windows-x86`, nightly), name `check-codegen.ps1`'s Generated-diff check as the primary guard for the regenerate-and-commit rule with the windows-x86 net48 compile as the second line, and document the manual windev fallback (isolated `C:\build` worktree) as the degraded-mode procedure with its trigger condition (SSH job red for infra reasons). If TST-25 is somehow delayed, do **not** hold these edits hostage — an interim commit correcting the docs to the two-job reality plus a mandatory manual cadence (per-merge for worker-touching changes, weekly otherwise) is strictly better than the current false claim; rewrite again when the SSH job lands.
+
+**Implementation.**
+1. `docs/GatewayTesting.md:426-431` — replace the `windows` and `live-mxaccess` bullets with: `windows-x86` (SSH-driven, per-push, x86 build + Worker.Tests on windev; failure semantics — red job = tier or change broken, how to distinguish) and the nightly scheduled job (full Worker.Tests + live-MXAccess smoke, `MXGATEWAY_RUN_LIVE_MXACCESS_TESTS=1`, never gates a push, on-failure issue notification). Reword the paragraph at `:433-437` so the Generated/ guard attribution is: primary = `check-codegen.ps1` regeneration diff (portable job), secondary = net48 compile (windows-x86).
+2. `scripts/check-codegen.ps1:17` — change "are guarded by the Windows CI job, not here" to "are guarded by the SSH-driven `windows-x86` CI job (see docs/GatewayTesting.md § Continuous Integration), not here".
+3. `.gitea/workflows/ci.yml:12-14` — cron comment now describes the nightly windev run (covered by TST-25 step 5); `:123-131` removal note replaced (TST-25 step 6).
+
+**Verification.** Greps prove the drift is gone: `grep -n '"windows"' docs/GatewayTesting.md` and `grep -n 'live-mxaccess' docs/GatewayTesting.md` return only descriptions of jobs that exist in `ci.yml`; `grep -n 'Windows CI job' scripts/check-codegen.ps1` returns the corrected text; `grep -rn 'definitive guard' docs/` attributes the Generated/ rule to `check-codegen.ps1`. Read the CI section back against the final `ci.yml` job list — every named job must exist, and every job in `ci.yml` must be named.
+
+---
+
+## TST-27 — `ShowTagValues` config row still says "Reserved"   `Medium` · `P1 (doc batch)` · Effort S
+
+**Finding.** `docs/GatewayConfiguration.md:185` reads "Reserved display control for tag values. The dashboard does not show full tag values by default." — but SEC-25 wired the flag live: `src/ZB.MOM.WW.MxGateway.Server/Dashboard/Hubs/DashboardEventBroadcaster.cs:16-29` blanks tag values from the deep-cloned event before mirroring to SignalR when `Dashboard:ShowTagValues` is `false` (the default). `docs/GatewayDashboardDesign.md:170` was updated in the SEC-25 change; the authoritative config reference was not — remediation-created drift, same-commit rule violated.
+
+**Impact.** An operator consulting the options table concludes toggling the flag does nothing, when it actually controls whether tag values reach **every** dashboard hub subscriber — security-relevant precisely because the per-session hub ACL (TST-15) does not exist yet: the redaction is currently the only thing between a low-trust Viewer and other sessions' tag values.
+
+**Design.** Documentation-only correction of the one stale row; no code change. State three things in the row: (1) what `false` (default) does — tag values are blanked from the `MxEvent` copy mirrored to the SignalR events hub; metadata (tag reference, quality, status, timestamps) still renders; (2) the security relevance — with no per-session hub ACL yet (TST-15), `true` exposes all sessions' tag values to every authenticated dashboard viewer; (3) the honest scope limit — the flag gates only the hub mirror, **not** the `/browse` live-value display (the TST-16 residual, tracked separately; do not paper over it by implying full coverage). Ship in the cycle's P1 doc-drift batch (with WRK-26, CLI-42, SEC residuals per the 00-overall roadmap) — one commit for the batch is fine since each edit is independently verifiable.
+
+**Implementation.**
+1. `docs/GatewayConfiguration.md:185` — replace the row description with the three-part text above, cross-referencing `docs/GatewayDashboardDesign.md` for the mechanism.
+2. While in the file, confirm the shape block's `"ShowTagValues": false` at `:60` needs no change (it doesn't — value and default are correct; only the prose row lies).
+3. No edit to `DashboardEventBroadcaster.cs` or `GatewayDashboardDesign.md` (both already correct).
+
+**Verification.** `grep -n 'Reserved' docs/GatewayConfiguration.md` no longer matches the `ShowTagValues` row; read the new row back against `DashboardEventBroadcaster.cs:16-29` (behavior), `Configuration/DashboardOptions.cs` (default `false`), and `GatewayDashboardDesign.md:170` (consistency). Confirm the row does **not** claim `/browse` coverage.
+
+---
+
+## TST-28 — Gateway-side `max_frame_bytes` untested in the CI-run suite   `Low` · `P2` · Effort S · relates IPC-02
+
+**Finding.** `src/ZB.MOM.WW.MxGateway.Server/Workers/WorkerClient.cs:988` populates `MaxFrameBytes = (uint)_connection.FrameOptions.MaxMessageBytes` in the `GatewayHello` (wired from `Worker.MaxMessageBytes` via `Sessions/SessionWorkerClientFactory.cs:73-76`), but no test under `src/ZB.MOM.WW.MxGateway.Tests` references `MaxFrameBytes` at all. The adoption side is asserted only in `Worker.Tests` — Windows-only, i.e. inside the TST-25 gap.
+
+**Impact.** A regression to sending `0` (which the worker deliberately treats as "older gateway, use default" per the IPC-02 negotiation contract) would pass every automated test while silently reverting the negotiated frame limit — an invisible downgrade of shipped IPC-02 behavior.
+
+**Design.** One cheap fake-worker assertion; no new infrastructure needed. `FakeWorkerHarness.CompleteStartupAsync` already reads and **returns** the `GatewayHello` envelope (`Tests/Gateway/Workers/Fakes/FakeWorkerHarness.cs:149-165`), and `CreateConnectedPairAsync` already accepts a `maxMessageBytes` override that flows into the `WorkerFrameProtocolOptions` shared with the `WorkerClient` the harness creates — so the test is: start the client, complete the handshake, assert on the returned envelope. Cover both the default (`WorkerFrameProtocolOptions.DefaultMaxMessageBytes`) and an overridden value (e.g. `2 * 1024 * 1024`) as a `[Theory]`, which also pins that the field tracks configuration rather than a constant. Rejected alternative: asserting via the worker-side adoption tests only — that is exactly the Windows-only blind spot this finding names.
+
+**Implementation.**
+1. Add a `[Theory]` to `src/ZB.MOM.WW.MxGateway.Tests/Gateway/Workers/WorkerClientTests.cs` (natural home — its `CompleteHandshakeAsync` helper at `:26` already drives this path), e.g. `StartAsync_SendsGatewayHelloWithConfiguredMaxFrameBytes`: create the harness with `CreateConnectedPairAsync(maxMessageBytes: value)` for `value` ∈ {default, 2 MiB}, create/start the `WorkerClient`, capture the envelope from `CompleteStartupAsync`, assert `envelope.GatewayHello.MaxFrameBytes == (uint)value` (and `!= 0` for readability of intent).
+2. No production code change; no doc change (`docs/WorkerFrameProtocol.md:25-30` already documents the negotiation).
+
+**Verification.** `dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter FullyQualifiedName~WorkerClientTests` green on macOS/Linux (i.e. it runs in the `portable` CI job — the point of the finding). Mutation check while developing: temporarily hard-code `MaxFrameBytes = 0` in `WorkerClient.cs:988` and confirm the new test fails, then revert.
+
+---
+
+## TST-29 — Retire `oldtasks.md`; delete root docs-review artifacts   `Low` · `P2` · Effort S
+
+**Finding.** TST-14's plan was "keep `oldtasks.md` only until the epic resumes, then delete", but TST-04 made it the durable governance record: `oldtasks.md:27-35,75-88` is now the only tracked statement that Phase 5 (orphan-worker reattach) is **deferred, not planned** and that `EnableOrphanReattach` (Task 26) "does not exist and must not be referenced". CLAUDE.md:79 links to it for exactly that statement. Meanwhile the mechanical half of TST-14 — deleting the five untracked, gitignored root working files (`MxAccessGateway-docs-{issues,fixed,final}.md`, `MxGatewayClient-docs-{issues,fixed}.md`, ~15k lines) — remains undone and keeps tripping greps.
+
+**Impact.** Low, but real: deleting `oldtasks.md` naively would orphan a load-bearing governance record; keeping it forever contradicts its own "just a human-readable mirror" charter (`oldtasks.md:12-14`) and leaves two sources that can drift from `docs/plans/2026-06-15-session-resilience.md.tasks.json`. The untracked artifacts are pure grep noise.
+
+**Design.** Move the durable decisions to where invariant-adjacent decisions live, then delete the mirror. `docs/DesignDecisions.md` gains a "Session-resilience epic scope" entry recording: Phase 5 deferred-not-planned with the rationale (reverses the "gateway restart does not reattach orphan workers" invariant; adoption-manifest store + phone-home protocol; deferred unless a concrete requirement appears); the explicit rule that `EnableOrphanReattach` does not exist and must not be referenced until its task lands; and the settled Phase-4 Viewer-default decision (admin-sees-all, Viewer strictly scoped to owned/granted sessions — `oldtasks.md:71-72`) so the TST-15 implementation doesn't have to rediscover it. The `tasks.json` remains the sole resume state for the still-pending Phase 4 tasks — one authority, no mirror. Do this now rather than waiting for the epic to close: the mirror's residual value (Task 14 Java remainder, Phase 4 checklist) is fully carried by `tasks.json` + the archreview register. Alternative rejected: keep `oldtasks.md` as the governance record — a file named "oldtasks" that must never be deleted is a trap for future cleanup passes.
+
+**Implementation.**
+1. `docs/DesignDecisions.md` — add the "Session-resilience epic scope" decision entry (content above, dated, citing the epic plan and TST-04).
+2. Update the two tracked references before deletion: CLAUDE.md:79 ("see `oldtasks.md` (session-resilience epic Phase 5)" → point at `docs/DesignDecisions.md`); `stillpending.md:7,165` ("remaining epic tasks tracked in `oldtasks.md`" → point at `docs/plans/2026-06-15-session-resilience.md.tasks.json`). Leave `docs/plans/2026-06-16-stillpending-section8-design.md` (historical plan; a stale pointer in a dated plan is acceptable — optionally annotate).
+3. `git rm oldtasks.md` — same commit as steps 1–2 (the record must never be absent from `main`).
+4. `rm` the five untracked root files: `MxAccessGateway-docs-{issues,fixed,final}.md`, `MxGatewayClient-docs-{issues,fixed}.md` (gitignored at `.gitignore:152-154` — deletion is local-only, zero repo impact; no commit involved).
+5. Do **not** touch `stillpending.md`'s own promote-or-migrate question, `REVIEW-PROCESS.md`, or `A2-galaxyrepository-adoption-handoff.md` — those remain TST-14's open triage in the prior-cycle doc.
+
+**Verification.** `grep -rn 'oldtasks' --include='*.md' .` (excluding `archreview/`, which is a historical record) returns no tracked references; `grep -n 'EnableOrphanReattach\|deferred, not planned' docs/DesignDecisions.md` finds the migrated statements; `ls *-docs-*.md` at the repo root returns nothing; `git status` clean. Read CLAUDE.md:79 back to confirm the invariant sentence still holds and its pointer resolves.
+
+---
+
+## Cross-domain dependencies
+
+- **TST-25 → old TST-05 / old TST-24:** the SSH-driven nightly is where the scheduled live-MXAccess smoke lands (closes old TST-05), and a working CI Windows tier unblocks wiring client wire-behavior tests into CI (old TST-24).
+- **TST-25 ↔ IPC-24/IPC-25:** the nightly windev job is also the natural home for any Windows-side codegen verification the contracts/IPC remediation adds; coordinate job naming so both plans extend the same `windows-x86`/nightly jobs rather than adding parallel ones.
+- **TST-26 ⊂ TST-25:** same commit, by rule.
+- **TST-27:** ships in the cycle's P1 doc-drift batch alongside WRK-26 and CLI-42 (roadmap item 8); its `/browse` residual stays with TST-16 (prior cycle).
+- **TST-28 ← IPC-02:** the test pins the gateway half of the IPC-02 negotiation; the worker half's tests become CI-run once TST-25 lands.