docs(archreview-p1): mark GWC-04 Done — Wave 3 complete

feat(gateway): decouple worker event enqueue from the read loop (GWC-04)
The read loop awaited EnqueueWorkerEventAsync inline, which blocks in the bounded event channel's timed WriteAsync (up to EventChannelFullModeTimeout, default 5s) when the channel is full with no/slow consumer. A WorkerCommandReply or heartbeat queued behind an event frame was then stalled, so an in-flight InvokeAsync could hit CommandTimeout even though the worker replied in time. Mirror the existing outbound WriteLoopAsync: add an unbounded event staging channel and a dedicated EventWriteLoopAsync. DispatchEnvelope is now fully synchronous — the WorkerEvent branch hands the event off with a non-blocking TryWrite and the read loop never awaits. The event write loop owns the timed WriteAsync into the bounded channel and the sustained-overflow ProtocolViolation fault (unchanged contract). Registered in WaitForBackgroundTasks + completed on close/fault/dispose. Test: reply arriving after events with a full, consumer-less event channel is dispatched promptly (no CommandTimeout) — pipe-harness, verified on windev. Docs: GatewayProcessDesign read/write/event-loop section.
2026-07-09 09:28:39 -04:00 · 2026-07-09 09:28:13 -04:00 · 2026-07-09 09:21:48 -04:00 · 2026-07-09 09:16:31 -04:00 · 2026-07-09 09:09:14 -04:00 · 2026-07-09 08:49:59 -04:00
41 changed files with 1368 additions and 149 deletions
@@ -88,7 +88,7 @@ powershell -ExecutionPolicy Bypass -File scripts/run-client-e2e-tests.ps1

 When source code changes, build and test the affected component before reporting work done. If the change crosses component boundaries, build each affected component — don't rely on a single top-level build:

-**Run targeted tests per task, never the full suite each time.** When executing a plan task-by-task, run only the tests that exercise the code that task touched (`dotnet test --filter "FullyQualifiedName~<TestClass>"`, or the per-task test named in the plan). The full gateway suite is slow and leaves orphaned testhost processes — run it at most once per phase (after a related batch of tasks lands), not after every task.
+**Run targeted tests per task, never the full suite each time.** When executing a plan task-by-task, run only the tests that exercise the code that task touched (`dotnet test --filter "FullyQualifiedName~<TestClass>"`, or the per-task test named in the plan). The full gateway suite is slow — run it at most once per phase (after a related batch of tasks lands), not after every task. This is a speed guideline, not a correctness one: the suite exits cleanly (verified on macOS and the Windows dev box — 0 surviving `testhost`/worker processes after a full run), so filtered runs are about turnaround, not about avoiding a process leak.

 | Changed area | Required verification |
 |---|---|
@@ -123,7 +123,7 @@ Gateway gRPC clients authenticate with an API key in metadata: `authorization: B

 Session event streaming is **owner-scoped**: the API key that opened a session is recorded on the session, and every `StreamEvents` attach/reattach is rejected with `PermissionDenied` unless the caller's key id matches the owner. Possessing the `event` scope and knowing a session id is not sufficient — this closes the reconnect/fan-out trust boundary (detach-grace and replay retention are on by default) so an `event`-scoped key cannot attach to another key's retained session.

-Dashboard auth is LDAP-backed (separate from the gRPC API-key model). `/login` binds against `MxGateway:Ldap` and maps the user's LDAP groups to `Admin` or `Viewer` via `MxGateway:Dashboard:GroupToRole`, then issues an HTTP-only secure `__Host-MxGatewayDashboard` cookie. SignalR hubs at `/hubs/{snapshot,alarms,events}` accept either the cookie or a 30-minute bearer minted at `/hubs/token`. `Dashboard:AllowAnonymousLocalhost` bypasses auth on loopback when enabled. `Dashboard:DisableLogin` (default `false`) auto-authenticates every dashboard request — including remote browsers — as `Dashboard:AutoLoginUser` (default `multi-role`) with both Admin and Viewer roles; dev/test only, never enable in production.
+Dashboard auth is LDAP-backed (separate from the gRPC API-key model). `/login` binds against `MxGateway:Ldap` and maps the user's LDAP groups to `Admin` or `Viewer` via `MxGateway:Dashboard:GroupToRole`, then issues an HTTP-only secure `__Host-MxGatewayDashboard` cookie. SignalR hubs at `/hubs/{snapshot,alarms,events}` accept either the cookie or a 5-minute bearer minted at `/hubs/token`. `Dashboard:AllowAnonymousLocalhost` (default `true`) grants **read-only** loopback access — it satisfies the Viewer requirement but never the Admin-only requirement, so anonymous localhost can view the dashboard but not reach API-key CRUD or session Close/Kill (`Authentication:Mode=Disabled` is scoped the same way). `Dashboard:DisableLogin` (default `false`) auto-authenticates every dashboard request — including remote browsers — as `Dashboard:AutoLoginUser` (default `multi-role`) with both Admin and Viewer roles; dev/test only, never enable in production.

 ## Process / Platform Notes

@@ -62,7 +62,7 @@ Full design + implementation for each row lives in the linked domain doc under i
 | GWC-01 | Critical | P0 | M | — | Done | Alarm monitor and distributor pump both drain the single worker event channel |
 | GWC-02 | High | P0 | M | — | Done | Faulted sessions are never swept |
 | GWC-03 | High | P0 | S | — | Done | Documented sparse-array max-length bound is unimplemented |
-| GWC-04 | Medium | P1 | M | — | Not started | Full event channel stalls the worker read loop behind command replies |
+| GWC-04 | Medium | P1 | M | — | Done | Full event channel stalls the worker read loop behind command replies |
 | GWC-05 | Medium | — | S | — | Not started | Worker pipe created with no ACL / no CurrentUserOnly |
 | GWC-06 | Medium | P2 | S | GWC-07,08 | Not started | Stopwatch allocated per streamed event |
 | GWC-07 | Medium | P2 | S | GWC-06,08 | Not started | Every mapped event is deep-cloned |
@@ -90,10 +90,10 @@ Full design + implementation for each row lives in the linked domain doc under i
 | WRK-01 | High | P0 | M | — | Done | Long `ReadBulk` self-faults as `StaHung`; all replies then dropped |
 | WRK-02 | Medium | — | M | — | Not started | STA thread death after startup is silent; future work hangs forever |
 | WRK-03 | Medium | — | S | WRK-02 | Not started | Commands after shutdown starts are dropped with no reply |
-| WRK-04 | Medium | — | M | — | Not started | Envelope `sequence` can appear out of order on the wire |
+| WRK-04 | Medium | — | M | — | Done | Envelope `sequence` can appear out of order on the wire |
 | WRK-05 | Medium | — | M | — | Not started | One transient alarm-poll failure kills the whole session |
 | WRK-06 | Medium | P2 | S | — | Not started | `MXSTATUS_PROXY` conversion reflects per field, per event |
-| WRK-07 | Medium | P1 | M | WRK-04 | Not started | Documented outbound write priority not implemented |
+| WRK-07 | Medium | P1 | M | WRK-04 | Done | Documented outbound write priority not implemented |
 | WRK-08 | Low | — | S | — | Not started | Residual event queue discarded at graceful shutdown, undocumented |
 | WRK-09 | Low | — | S | — | Not started | No `AppDomain.UnhandledException` hook |
 | WRK-10 | Low | — | S | — | Not started | Top-level catch logs exception type but never message |
@@ -113,9 +113,9 @@ Full design + implementation for each row lives in the linked domain doc under i
 | ID | Sev | Tier | Eff | Dep | Status | Title |
 |---|---|:-:|:-:|---|---|---|
 | IPC-01 | High | P1 | M | — | Done | Published client descriptor set 7 weeks stale; nothing enforces freshness |
-| IPC-02 | Medium | P1 | M | IPC-03 | Not started | Worker max frame size hard-coded, cannot follow gateway config |
-| IPC-03 | Medium | P1 | M | IPC-02 | Not started | gRPC max == pipe max with zero headroom; oversized write faults whole session |
-| IPC-04 | Medium | P1 | S | IPC-03 | Not started | `DrainEvents max_events=0` packs entire queue into one reply frame |
+| IPC-02 | Medium | P1 | M | IPC-03 | Done | Worker max frame size hard-coded, cannot follow gateway config |
+| IPC-03 | Medium | P1 | M | IPC-02 | Done | gRPC max == pipe max with zero headroom; oversized write faults whole session |
+| IPC-04 | Medium | P1 | S | IPC-03 | Done | `DrainEvents max_events=0` packs entire queue into one reply frame |
 | IPC-05 | Medium | P2 | S | — | Not started | Redundant deep copies on command and event hot paths |
 | IPC-06 | Medium | P2 | S | — | Not started | `gateway.md` Worker Envelope sketch no longer matches the contract |
 | IPC-07 | Medium | P2 | S | — | Not started | `docs/Grpc.md` says six RPCs; there are seven |
@@ -140,7 +140,7 @@ Full design + implementation for each row lives in the linked domain doc under i
 | ID | Sev | Tier | Eff | Dep | Status | Title |
 |---|---|:-:|:-:|---|---|---|
 | SEC-01 | Medium | P1 | M | — | Done | Windows-absolute default paths become relative files off-Windows |
-| SEC-02 | Medium | P1 | S | — | Not started | `AllowAnonymousLocalhost` satisfies AdminOnly, not just Viewer |
+| SEC-02 | Medium | P1 | S | — | Done | `AllowAnonymousLocalhost` satisfies AdminOnly, not just Viewer |
 | SEC-03 | Medium | P2 | S | — | Not started | Dashboard cookie lost its `__Host-` prefix; four docs still promise it |
 | SEC-04 | Medium | P1 | S | SEC-03 | Done | `DisableLogin` has no production guard |
 | SEC-05 | Medium | P1 | M | — | Done | Hub bearer tokens irrevocable for 30 min, carried in query string |
@@ -150,7 +150,7 @@ Full design + implementation for each row lives in the linked domain doc under i
 | SEC-09 | Medium | P2 | S | — | Not started | Dashboard design-doc `GroupToRole` sample now fails startup validation |
 | SEC-10 | Medium | P1 | L | — | Done | No API-key expiry |
 | SEC-11 | Medium | P1 | M | SEC-08 | Done | No rate limiting or lockout on either auth surface |
-| SEC-12 | Medium | P1 | M | — | Not started | Dashboard Close/Kill bypass the canonical audit store |
+| SEC-12 | Medium | P1 | M | — | Done | Dashboard Close/Kill bypass the canonical audit store |
 | SEC-13 | Low | — | S | SEC-30 | Not started | Redactor's credential-command list omits secured-bulk variants |
 | SEC-14 | Low | — | S | SEC-02,20 | Not started | `/metrics` + `/health` unauthenticated; a metric leaks session ids |
 | SEC-15 | Low | — | S | — | Not started | GET `/logout` skips antiforgery |
@@ -158,7 +158,7 @@ Full design + implementation for each row lives in the linked domain doc under i
 | SEC-17 | Low | — | S | — | Not started | Interceptor does not override client-streaming/duplex handlers |
 | SEC-18 | Low | — | S | — | Not started | Pepper-unavailable detection matches library message text |
 | SEC-19 | Low | — | S | — | Not started | Canonical audit store re-issues `CREATE TABLE` per write/read |
-| SEC-20 | Low | P1 | S | — | Not started | `session_id` metric tag is unbounded cardinality |
+| SEC-20 | Low | P1 | S | — | Done | `session_id` metric tag is unbounded cardinality |
 | SEC-21 | Low | — | M | — | Not started | Snapshot publisher works every second regardless of audience |
 | SEC-22 | Low | P2 | M | — | Not started | `docs/Authentication.md` documents types no longer in this repo |
 | SEC-23 | Low | — | S | SEC-03,04 | Not started | Validator ignores `DisableLogin`/`AutoLoginUser`/`RequireHttpsCookie`/`CookieName` |
@@ -220,7 +220,7 @@ Full design + implementation for each row lives in the linked domain doc under i
 | TST-05 | Medium | P1 | S | TST-03 | Not started | Real-worker control/COM paths verified opt-in only |
 | TST-06 | Medium | — | M | — | Not started | Dashboard live-data path untested |
 | TST-07 | Medium | — | S | — | Not started | Real-clock sleeps with negative assertions are latent flakes |
-| TST-08 | Medium | P1 | M | — | Not started | Full-suite orphaned testhost processes |
+| TST-08 | Medium | P1 | M | — | Done | Full-suite orphaned testhost processes (does not reproduce; doc de-stale) |
 | TST-09 | Medium | — | M | — | Not started | Health checks cover only the auth store |
 | TST-10 | Medium | — | M | — | Not started | Deployment/upgrade undocumented and hand-rolled |
 | TST-11 | Medium | P2 | M | — | Not started | No version discipline on server; client versions drift |
@@ -255,6 +255,11 @@ Findings the review flagged as one coordinated design pass — sequence them tog
 |---|---|
 | 2026-07-09 | Initial tracking doc generated from the six domain remediation designs. All 153 findings `Not started`. |
 | 2026-07-09 | P0 tier executed via parallel agents. GWC-01/02/03, TST-02, TST-12, CLI-01, CLI-03 → `Done` (NonWindows build clean; gateway 135/135 targeted tests pass, Go `go test ./...` + `-race` pass, Rust `test`/`clippy -D warnings` pass). |
+| 2026-07-09 | Windows full-suite temp-file-lock flakiness (discovered via TST-08) → **fixed** (commit `11a716a`). Two shared-state parallel collisions that macOS never surfaces: (1) self-signed cert generation — `SelfSignedCertificateProvider` now stages the PFX in a unique `<path>.<guid>.tmp` (was a fixed name that concurrent/interrupted writers collided on); `TestHostEnvironmentInitializer` defaults the cert path to a per-process temp dir (so the suite never writes shared `ProgramData` or fights the deployed service); `GatewayTlsBootstrapTests` moved to a `DisableParallelization` collection so its process-global env-var mutation can't bleed into parallel tests. (2) `AuthStoreHealthCheckTests` — reuse the existing `TempDatabaseDirectory` helper (`SqliteConnection.ClearAllPools()` before delete) so the Microsoft.Data.Sqlite pool releases the temp `.db` handle. **windev-verified:** 3 consecutive full-suite runs all **793 passed / 2 failed, 0 new orphans** (was flaky 2–4 fails, count-varying). The 2 stable remaining failures are pre-existing windev-environmental (`SelfSignedCertificateProviderTests.GenerateCertificate_HasExpectedSansEkuAndValidity` machine-name/FQDN SAN assertion; `EventStreamServiceTests…TracksAggregateQueueDepth` timing) — both pass on macOS, unrelated to this change. |
+| 2026-07-09 | P1 TST-08 (orphaned testhost) → `Done` by evidence: the claimed leak does **not** reproduce. Full `ZB.MOM.WW.MxGateway.Tests` suite exits cleanly on macOS (0 survivors) and on the Windows dev box (windev, `origin/main` worktree: 0 new `testhost` and 0 new `ZB.MOM.WW.MxGateway.Worker` processes after a full run, isolated by StartTime). Static audit confirms the prime-suspect fixtures already dispose deterministically (WorkerClient cancels its `_stopCts` + awaits read/write/heartbeat tasks with a 5 s timeout; GatewaySession disposes the client; E2E/harness fixtures use `await using`). CLAUDE.md "Source Update Workflow" updated to drop the stale "leaves orphaned testhost processes" rationale (filtered runs kept as a speed guideline). **Discovered separately (NOT TST-08, still open):** Windows-only temp-file-lock flakiness in ~4 full-host test classes — `AuthStoreHealthCheckTests` (Microsoft.Data.Sqlite connection-pool holds the temp `.db` handle after `await using`, so the finally `File.Delete` throws "used by another process"), `GatewayTlsBootstrapTests`/`DashboardCookieOptionsTests`/`DashboardHubsRegistrationTests` (a `gw.pfx.tmp` handle from `SelfSignedCertificateProvider`'s fixed-name `path+".tmp"` atomic-write racing teardown). Count varies run-to-run (2–4 fails); macOS never sees it (Unix allows deleting open files). Fix = `SqliteConnection.ClearAllPools()` before temp delete + a unique/guarded `.tmp` name; verify on windev. |
+| 2026-07-09 | P1 Wave 3 (event-channel decoupling). GWC-04 → `Done`. `WorkerClient`'s read loop awaited `EnqueueWorkerEventAsync` inline, which blocks in the bounded event channel's timed `WriteAsync` (≤ `EventChannelFullModeTimeout`, 5 s) when the channel is full with a slow/absent `StreamEvents` consumer — stalling any `WorkerCommandReply`/heartbeat queued behind an event frame, so an in-flight `InvokeAsync` could hit `CommandTimeout` despite a timely worker reply. Fix mirrors the existing outbound `WriteLoopAsync`: an unbounded event **staging** channel + a dedicated `EventWriteLoopAsync`; `DispatchEnvelope` is now fully synchronous and the `WorkerEvent` branch hands off with a non-blocking `TryWrite`, so the read loop never awaits event enqueue. The event write loop owns the timed write + the sustained-overflow `ProtocolViolation` fault (unchanged contract); registered in `WaitForBackgroundTasks`, completed on close/fault/dispose. Server build clean (0 warnings); non-pipe event tests 37/38 (the 1 is the known parallel-load `EventStreamServiceTests…TracksAggregateQueueDepth` timing flake — passes in isolation). New pipe-harness test (reply after events with a full consumer-less channel dispatches without `CommandTimeout`) verified on windev. Docs: GatewayProcessDesign read/write/event-loop section. Commit `32d6b48`. **Wave 3 complete.** |
+| 2026-07-09 | P1 Wave 3 (size/backpressure topology + write ordering). IPC-02/03/04 + WRK-04/07 → `Done`. **Size negotiation (IPC-02):** added `GatewayHello.max_frame_bytes` (regen `Generated/` + `clients/proto` descriptor refresh with pinned protoc 34.1); the gateway sends its negotiated worker-frame max and the worker adopts it (`WorkerFrameProtocolOptions.AdoptNegotiatedMaxMessageBytes`, 0 = keep default, >256 MiB rejected) instead of a hard-coded default. **Headroom (IPC-03):** the pipe frame max now sits `EnvelopeOverheadReserveBytes` (64 KiB) above the public gRPC cap (default `Worker.MaxMessageBytes` 16 MiB→16 MiB+64 KiB), cross-validated at startup; `WorkerClient` pre-checks command envelope size and fails only the offending correlation (`ResourceExhausted`) instead of `SetFaulted`ing the session. **Drain bound (IPC-04):** gateway request validator rejects `DrainEvents max_events` above 10 000; the worker caps each reply at `MaxDrainEventsPerReply` (10 000) and treats `max_events = 0` as that cap, not "drain all". **Sequence (WRK-04):** `WorkerFrameWriter` stamps the envelope `Sequence` at the point of writing under the write lock, so wire order and stamped sequence always agree under concurrent producers. **Priority (WRK-07):** the worker writer is now a cooperative priority scheduler — control frames (reply/fault/heartbeat/shutdown-ack) drain ahead of event frames; per-frame validation/size rejections fail only that frame, a stream failure fails all queued. Docs same-change (GatewayConfiguration, WorkerFrameProtocol, gateway.md). **Verified:** macOS NonWindows build clean + validator/grpc tests green; **windev** x86 worker builds clean, `Worker.Tests` 352 passed / 0 failed / 11 skipped (incl. new monotonic-sequence, control-before-event priority, negotiated-max, drain-bound tests), gateway `Tests` 799 passed / 3 failed — all 3 pre-existing windev-environmental (SelfSigned SAN + 2 `EventStreamServiceTests` timing, both pass in isolation). Commits `c8b3a22` (gateway half), `ebe6aea` (worker half), `309296f` (descriptor + default-expectation refresh). GWC-04 (event-channel decoupling) is the remaining Wave 3 item. |
+| 2026-07-09 | P1 S-misc (dashboard/observability hardening). SEC-02/12/20 → `Done`. SEC-02: `DashboardAuthorizationHandler` restricts the loopback + `Authentication:Mode=Disabled` bypasses to read-only (they satisfy a Viewer-bearing requirement but never `AdminOnly`), closing the policy-layer gap where anonymous localhost was authorized for Admin surfaces. SEC-12: `DashboardSessionAdminService` now emits canonical `AuditEvent`s (`dashboard-close-session`/`dashboard-kill-worker`, category `SessionAdmin`) through `IAuditWriter` on Success/Failure/Denied, so Close/Kill land durable audit rows. SEC-20: dropped the unbounded `session_id` tag from the exported `mxgateway.heartbeats.failed` counter. Docs updated same-change (CLAUDE.md, GatewayDashboardDesign.md, Metrics.md). Server build clean (0 warnings); targeted classes 30/30 pass; broader Dashboard+Security+GatewayApplication+Metrics sweep 295/295 pass. |
 | 2026-07-09 | P1 Wave 2b (security authz+hub). SEC-05/07/08/11 → `Done` (hub-token lifetime; QueryActiveAlarms scope arm; gateway-side verification cache + last-used coalescing; login rate limit + per-peer gRPC failure limiter). Full-suite checkpoint caught + fixed regressions the earlier narrow SEC-01/04/06 filter missed: cross-platform path-rooting, an `IHostEnvironment` fallback for minimal DI containers, a test-assembly `ASPNETCORE_ENVIRONMENT=Development` default, and a platform-correct default-path assertion. Suite: 747 passed / 42 failed, all 42 pre-existing macOS named-pipe-harness env failures. |
 | 2026-07-09 | P1 Wave 2a (security). SEC-01/04/06 → `Done` (config path-rooting + production validator guards; Server build clean, validator+hygiene tests 53/53). SEC-10 → `Done`: the shared `ZB.MOM.WW.Auth.ApiKeys` gained optional `ExpiresUtc` (expired keys rejected, auth DB auto-migrates to schema v3) via a concurrent HistorianGateway-remediation session's "G-2"; this repo consumes it by bumping the four `Auth.*` refs 0.1.2→0.1.4 (commit 197731a). Remaining SEC-10 polish (`apikey create --expires` + dashboard staleness badge) tracked as a small follow-up. |
 | 2026-07-09 | P1 Wave 1 (CI + codegen freshness + Rust buildability) via parallel agents. IPC-01/09/19/20, CLI-02 → `Done`; TST-03 → `In review` (CI pipeline authored + YAML/layout-validated, but not yet executed on a Gitea runner — proven on first push). Verified on macOS: NonWindows build clean, `ClientProtoInputTests` 5/5, `publish-client-proto-inputs.ps1 -Check` exit 0, `cargo package` (no `--no-verify`) compiles standalone. Added a vendored-Rust-proto drift guard (Check 3) to `check-codegen.ps1` closing the CLI-02 static-copy risk. IPC-09 script guards not executed end-to-end (need pinned python/JRE toolchains); logic is PATH-resolution + version assertion. |
@@ -29,7 +29,7 @@ paths, timeouts, queue sizes, enum values, or protocol values are invalid.
      "ShutdownTimeoutSeconds": 10,
      "HeartbeatIntervalSeconds": 5,
      "HeartbeatGraceSeconds": 15,
-      "MaxMessageBytes": 16777216
+      "MaxMessageBytes": 16842752
    },
    "Sessions": {
      "DefaultCommandTimeoutSeconds": 30,
@@ -117,12 +117,16 @@ CWD (SEC-01).
 | `MxGateway:Worker:ShutdownTimeoutSeconds` | `10` | Grace period for worker shutdown before the gateway treats shutdown as failed and may kill the worker process tree. |
 | `MxGateway:Worker:HeartbeatIntervalSeconds` | `5` | Worker heartbeat send interval and gateway heartbeat check cadence input. |
 | `MxGateway:Worker:HeartbeatGraceSeconds` | `15` | Maximum age of the last worker heartbeat before the gateway faults the worker. This must be greater than or equal to `HeartbeatIntervalSeconds`. |
-| `MxGateway:Worker:MaxMessageBytes` | `16777216` | Maximum worker IPC frame payload size in bytes. The validator allows values from `1024` through `268435456`. |
+| `MxGateway:Worker:MaxMessageBytes` | `16842752` | Maximum worker IPC frame payload size in bytes. The validator allows values from `1024` through `268435456`, and additionally requires this to be at least `MxGateway:Protocol:MaxGrpcMessageBytes` plus a 65536-byte (64 KiB) envelope-overhead reserve. The default is the 16 MiB public gRPC cap plus that reserve. The gateway conveys the negotiated value to the worker in the handshake (`GatewayHello.max_frame_bytes`), so the worker frames to the same limit rather than a hard-coded default. |

 `StartupProbeRetryAttempts`, `StartupProbeRetryDelayMilliseconds`,
 `PipeConnectAttemptTimeoutMilliseconds`, timeout values, heartbeat values, and
 `MaxMessageBytes` must be positive. `MaxMessageBytes` is intentionally bounded
-to avoid accidental large allocations from malformed or oversized frames.
+to avoid accidental large allocations from malformed or oversized frames. It
+must also stay above `MaxGrpcMessageBytes` by the envelope-overhead reserve so a
+maximally-sized accepted gRPC payload always fits one worker frame once wrapped
+in a `WorkerEnvelope`; otherwise the oversized outbound write would fault the
+whole session. Startup validation fails fast when the headroom is violated.

 ## Session Options

@@ -256,7 +260,7 @@ dev→production hardening posture.
 | Option | Default | Description |
 |--------|---------|-------------|
 | `MxGateway:Protocol:WorkerProtocolVersion` | `1` | Worker IPC protocol version expected by the gateway and worker. This must match `GatewayContractInfo.WorkerProtocolVersion`. |
-| `MxGateway:Protocol:MaxGrpcMessageBytes` | `16777216` | Public gRPC max send and receive message size in bytes. The same default is used by official clients. The validator allows values from `1024` through `268435456`. |
+| `MxGateway:Protocol:MaxGrpcMessageBytes` | `16777216` | Public gRPC max send and receive message size in bytes. The same default is used by official clients. The validator allows values from `1024` through `268435456`, and requires `MxGateway:Worker:MaxMessageBytes` to exceed this by at least the 64 KiB worker-frame envelope reserve (see the Worker options). |

 The protocol option is exposed for diagnostics and explicit deployment
 configuration, not for compatibility negotiation. A mismatch fails validation
@@ -257,6 +257,15 @@ gated by a confirmation dialog before it reaches `ISessionManager`.
  is killed immediately with no graceful-shutdown attempt. The session is
  removed from the registry and the open-session slot is released either way.

+Both actions write a canonical `AuditEvent` through `IAuditWriter` into the
+`audit_event` store (category `SessionAdmin`, actions `dashboard-close-session`
+and `dashboard-kill-worker`) in addition to the operational `ILogger` line — so a
+worker killed mid-production leaves a durable, queryable row rather than only a
+rotatable log entry. The event records the LDAP actor, the session id (`Target`),
+the remote address, and an outcome of `Success`, `Failure`, or `Denied`
+(an unauthorized attempt is still audited as `Denied`). This mirrors the API-key
+management audit path.
+
 ### Workers page

 Show:
@@ -436,11 +445,16 @@ Three authorization policies are registered:
  cookie OR a `MxGateway.Dashboard.HubToken` bearer (used by WebSocket upgrades
  where the cookie can't be forwarded).

-Two environmental bypasses still apply: `MxGateway:Authentication:Mode = Disabled`
-authorizes every request, and `MxGateway:Dashboard:AllowAnonymousLocalhost`
-(default `true`) authorizes any loopback request without a role check. Remote
-requests always require an authenticated principal carrying at least the
-Viewer role.
+Two environmental bypasses still apply, both scoped to **read-only** access:
+`MxGateway:Authentication:Mode = Disabled` and `MxGateway:Dashboard:AllowAnonymousLocalhost`
+(default `true`, loopback only) each satisfy a requirement that includes the Viewer
+role (`AnyDashboardRole`) but **never** the `AdminOnly` requirement. Destructive/admin
+surfaces (API-key CRUD, session Close, worker Kill) still demand a real `Administrator`
+role claim, so the "anonymous localhost is read-only" contract holds at the policy layer
+rather than only in downstream service re-checks. Remote requests (outside the `Disabled`
+bypass) always require an authenticated principal carrying at least the Viewer role. The
+loopback test trusts `Connection.RemoteIpAddress`; if forwarded-headers middleware is ever
+added upstream, `DashboardAuthorizationHandler.IsLoopbackRequest()` must be revisited.

 ### DisableLogin dev bypass

@@ -481,7 +481,9 @@ Internally it owns:
 - pipe stream,
 - read loop,
 - write loop,
+- event write loop,
 - outbound command/control channel serialized by the write loop,
+- unbounded event staging channel drained by the event write loop,
 - bounded inbound event channel,
 - pending command dictionary keyed by correlation id,
 - heartbeat monitor,
@@ -499,15 +501,29 @@ The read loop:
 1. Reads one frame.
 2. Parses `WorkerEnvelope`.
 3. Validates protocol fields.
-4. Dispatches by body type:
+4. Dispatches by body type, **synchronously and without blocking**:
   - `WorkerCommandReply`: completes pending command.
-   - `WorkerEvent`: enqueues event.
+   - `WorkerEvent`: hands the event to the event write loop via a non-blocking staging write.
   - `WorkerHeartbeat`: updates heartbeat timestamp.
   - `WorkerFault`: faults session.
 5. Stops when pipe closes or cancellation is requested.

 If the pipe closes while the session is not closing, fault the session.

+The read loop never awaits event enqueue. Events are staged to the event write
+loop with a non-blocking write, so a full inbound event channel (a slow or
+absent `StreamEvents` consumer) cannot stall the read loop behind an event and
+delay a command reply or heartbeat (GWC-04). The bounded-channel backpressure
+window (`EventChannelFullModeTimeout`) and the sustained-overflow fault are
+applied by the event write loop, not the read loop.
+
+### Event write loop
+
+The event write loop drains the staging channel and performs the timed write
+into the bounded inbound event channel. When the inbound channel stays full past
+`EventChannelFullModeTimeout` it faults the session (`ProtocolViolation`) — the
+same overflow contract as before, moved off the read loop.
+
 ### Write loop

 The write loop serializes all writes to the pipe. No other code should write to
@@ -44,7 +44,7 @@ All counters are `Counter<long>`. Tag values come from the call sites listed und
 | `mxgateway.faults` | `category` | Faults reported by session, event, or worker code paths. The category is a `SessionManagerErrorCode` or `WorkerClientErrorCode` name. |
 | `mxgateway.workers.killed` | `reason` | Forced terminations of worker processes. |
 | `mxgateway.workers.exited` | `reason` | Clean or fault-driven worker exits. |
-| `mxgateway.heartbeats.failed` | `session_id` | Worker heartbeat misses tracked per session. |
+| `mxgateway.heartbeats.failed` | *(none)* | Aggregate worker heartbeat misses. The exported counter carries **no** `session_id` tag — per-session attribution is unbounded cardinality (every session mints a new time series), so it is kept out of the exporter; use the dashboard snapshot or the log scope for per-session detail. |
 | `mxgateway.grpc.streams.disconnected` | `reason` | Detachments of the dashboard or client gRPC event stream. |
 | `mxgateway.retries.attempted` | `area` | Resilience retries executed by gateway components. |

@@ -17,8 +17,17 @@ payload_length bytes protobuf WorkerEnvelope
 ```

 The reader rejects zero-length payloads and payloads larger than the configured
-maximum before allocating the payload buffer. The default maximum is 16 MiB,
-matching the gateway process design.
+maximum before allocating the payload buffer. The default maximum is the 16 MiB
+public gRPC cap plus a 64 KiB envelope-overhead reserve (16842752 bytes) so a
+maximally-sized accepted gRPC payload always fits one worker frame once wrapped
+in a `WorkerEnvelope`.
+
+The gateway is the source of truth for this maximum: it conveys the negotiated
+value in the handshake as `GatewayHello.max_frame_bytes`, and the worker adopts
+it as its `WorkerFrameProtocolOptions.MaxMessageBytes` instead of a hard-coded
+default. A `max_frame_bytes` of 0 (an older gateway that never set the field)
+means "use the worker's built-in default". This keeps both ends framing to the
+same limit rather than depending on matched compile-time constants.

 ## Envelope Validation

@@ -424,7 +424,10 @@ Optional diagnostics:
 - `Ping`
 - `GetSessionState`
 - `GetWorkerInfo`
- `DrainEvents`
+- `DrainEvents` — diagnostic; `max_events` is bounded (the gateway rejects requests
+  above a public ceiling, and the worker caps each reply at its own per-reply limit,
+  treating `max_events = 0` as "the default cap") so one drain cannot pack an
+  unbounded, session-killing reply frame.
 - `ShutdownWorker`

 Do not compress MXAccess semantics into generic verbs too early. A command enum
@@ -44,63 +44,64 @@ namespace ZB.MOM.WW.MxGateway.Contracts.Proto {
            "Y2Nlc3Nfd29ya2VyLnYxLldvcmtlckV2ZW50SAASPwoQd29ya2VyX2hlYXJ0",
            "YmVhdBgTIAEoCzIjLm14YWNjZXNzX3dvcmtlci52MS5Xb3JrZXJIZWFydGJl",
            "YXRIABI3Cgx3b3JrZXJfZmF1bHQYFCABKAsyHy5teGFjY2Vzc193b3JrZXIu",
-            "djEuV29ya2VyRmF1bHRIAEIGCgRib2R5IloKDEdhdGV3YXlIZWxsbxIiChpz",
+            "djEuV29ya2VyRmF1bHRIAEIGCgRib2R5InMKDEdhdGV3YXlIZWxsbxIiChpz",
            "dXBwb3J0ZWRfcHJvdG9jb2xfdmVyc2lvbhgBIAEoDRINCgVub25jZRgCIAEo",
-            "CRIXCg9nYXRld2F5X3ZlcnNpb24YAyABKAkiaQoLV29ya2VySGVsbG8SGAoQ",
-            "cHJvdG9jb2xfdmVyc2lvbhgBIAEoDRINCgVub25jZRgCIAEoCRIZChF3b3Jr",
-            "ZXJfcHJvY2Vzc19pZBgDIAEoBRIWCg53b3JrZXJfdmVyc2lvbhgEIAEoCSKO",
-            "AQoLV29ya2VyUmVhZHkSGQoRd29ya2VyX3Byb2Nlc3NfaWQYASABKAUSFwoP",
-            "bXhhY2Nlc3NfcHJvZ2lkGAIgASgJEhYKDm14YWNjZXNzX2Nsc2lkGAMgASgJ",
-            "EjMKD3JlYWR5X3RpbWVzdGFtcBgEIAEoCzIaLmdvb2dsZS5wcm90b2J1Zi5U",
-            "aW1lc3RhbXAidwoNV29ya2VyQ29tbWFuZBIvCgdjb21tYW5kGAEgASgLMh4u",
-            "bXhhY2Nlc3NfZ2F0ZXdheS52MS5NeENvbW1hbmQSNQoRZW5xdWV1ZV90aW1l",
-            "c3RhbXAYAiABKAsyGi5nb29nbGUucHJvdG9idWYuVGltZXN0YW1wIoEBChJX",
-            "b3JrZXJDb21tYW5kUmVwbHkSMgoFcmVwbHkYASABKAsyIy5teGFjY2Vzc19n",
-            "YXRld2F5LnYxLk14Q29tbWFuZFJlcGx5EjcKE2NvbXBsZXRlZF90aW1lc3Rh",
-            "bXAYAiABKAsyGi5nb29nbGUucHJvdG9idWYuVGltZXN0YW1wIh4KDFdvcmtl",
-            "ckNhbmNlbBIOCgZyZWFzb24YASABKAkiUQoOV29ya2VyU2h1dGRvd24SLwoM",
-            "Z3JhY2VfcGVyaW9kGAEgASgLMhkuZ29vZ2xlLnByb3RvYnVmLkR1cmF0aW9u",
-            "Eg4KBnJlYXNvbhgCIAEoCSJIChFXb3JrZXJTaHV0ZG93bkFjaxIzCgZzdGF0",
-            "dXMYASABKAsyIy5teGFjY2Vzc19nYXRld2F5LnYxLlByb3RvY29sU3RhdHVz",
-            "IjoKC1dvcmtlckV2ZW50EisKBWV2ZW50GAEgASgLMhwubXhhY2Nlc3NfZ2F0",
-            "ZXdheS52MS5NeEV2ZW50IqUCCg9Xb3JrZXJIZWFydGJlYXQSGQoRd29ya2Vy",
-            "X3Byb2Nlc3NfaWQYASABKAUSLgoFc3RhdGUYAiABKA4yHy5teGFjY2Vzc193",
-            "b3JrZXIudjEuV29ya2VyU3RhdGUSPwobbGFzdF9zdGFfYWN0aXZpdHlfdGlt",
-            "ZXN0YW1wGAMgASgLMhouZ29vZ2xlLnByb3RvYnVmLlRpbWVzdGFtcBIdChVw",
-            "ZW5kaW5nX2NvbW1hbmRfY291bnQYBCABKA0SIgoab3V0Ym91bmRfZXZlbnRf",
-            "cXVldWVfZGVwdGgYBSABKA0SGwoTbGFzdF9ldmVudF9zZXF1ZW5jZRgGIAEo",
-            "BBImCh5jdXJyZW50X2NvbW1hbmRfY29ycmVsYXRpb25faWQYByABKAki9AEK",
-            "C1dvcmtlckZhdWx0EjkKCGNhdGVnb3J5GAEgASgOMicubXhhY2Nlc3Nfd29y",
-            "a2VyLnYxLldvcmtlckZhdWx0Q2F0ZWdvcnkSFgoOY29tbWFuZF9tZXRob2QY",
-            "AiABKAkSFAoHaHJlc3VsdBgDIAEoBUgAiAEBEhYKDmV4Y2VwdGlvbl90eXBl",
-            "GAQgASgJEhoKEmRpYWdub3N0aWNfbWVzc2FnZRgFIAEoCRI8Cg9wcm90b2Nv",
-            "bF9zdGF0dXMYBiABKAsyIy5teGFjY2Vzc19nYXRld2F5LnYxLlByb3RvY29s",
-            "U3RhdHVzQgoKCF9ocmVzdWx0KpcCCgtXb3JrZXJTdGF0ZRIcChhXT1JLRVJf",
-            "U1RBVEVfVU5TUEVDSUZJRUQQABIZChVXT1JLRVJfU1RBVEVfU1RBUlRJTkcQ",
-            "ARIcChhXT1JLRVJfU1RBVEVfSEFORFNIQUtJTkcQAhIhCh1XT1JLRVJfU1RB",
-            "VEVfSU5JVElBTElaSU5HX1NUQRADEhYKEldPUktFUl9TVEFURV9SRUFEWRAE",
-            "EiIKHldPUktFUl9TVEFURV9FWEVDVVRJTkdfQ09NTUFORBAFEh4KGldPUktF",
-            "Ul9TVEFURV9TSFVUVElOR19ET1dOEAYSGAoUV09SS0VSX1NUQVRFX1NUT1BQ",
-            "RUQQBxIYChRXT1JLRVJfU1RBVEVfRkFVTFRFRBAIKscEChNXb3JrZXJGYXVs",
-            "dENhdGVnb3J5EiUKIVdPUktFUl9GQVVMVF9DQVRFR09SWV9VTlNQRUNJRklF",
-            "RBAAEisKJ1dPUktFUl9GQVVMVF9DQVRFR09SWV9JTlZBTElEX0FSR1VNRU5U",
-            "UxABEjcKM1dPUktFUl9GQVVMVF9DQVRFR09SWV9HQVRFV0FZX0FVVEhFTlRJ",
-            "Q0FUSU9OX0ZBSUxFRBACEisKJ1dPUktFUl9GQVVMVF9DQVRFR09SWV9QUk9U",
-            "T0NPTF9NSVNNQVRDSBADEiwKKFdPUktFUl9GQVVMVF9DQVRFR09SWV9QUk9U",
-            "T0NPTF9WSU9MQVRJT04QBBIrCidXT1JLRVJfRkFVTFRfQ0FURUdPUllfUElQ",
-            "RV9ESVNDT05ORUNURUQQBRIyCi5XT1JLRVJfRkFVTFRfQ0FURUdPUllfTVhB",
-            "Q0NFU1NfQ1JFQVRJT05fRkFJTEVEEAYSMQotV09SS0VSX0ZBVUxUX0NBVEVH",
-            "T1JZX01YQUNDRVNTX0NPTU1BTkRfRkFJTEVEEAcSOgo2V09SS0VSX0ZBVUxU",
-            "X0NBVEVHT1JZX01YQUNDRVNTX0VWRU5UX0NPTlZFUlNJT05fRkFJTEVEEAgS",
-            "IgoeV09SS0VSX0ZBVUxUX0NBVEVHT1JZX1NUQV9IVU5HEAkSKAokV09SS0VS",
-            "X0ZBVUxUX0NBVEVHT1JZX1FVRVVFX09WRVJGTE9XEAoSKgomV09SS0VSX0ZB",
-            "VUxUX0NBVEVHT1JZX1NIVVRET1dOX1RJTUVPVVQQC0ImqgIjWkIuTU9NLldX",
-            "Lk14R2F0ZXdheS5Db250cmFjdHMuUHJvdG9iBnByb3RvMw=="));
+            "CRIXCg9nYXRld2F5X3ZlcnNpb24YAyABKAkSFwoPbWF4X2ZyYW1lX2J5dGVz",
+            "GAQgASgNImkKC1dvcmtlckhlbGxvEhgKEHByb3RvY29sX3ZlcnNpb24YASAB",
+            "KA0SDQoFbm9uY2UYAiABKAkSGQoRd29ya2VyX3Byb2Nlc3NfaWQYAyABKAUS",
+            "FgoOd29ya2VyX3ZlcnNpb24YBCABKAkijgEKC1dvcmtlclJlYWR5EhkKEXdv",
+            "cmtlcl9wcm9jZXNzX2lkGAEgASgFEhcKD214YWNjZXNzX3Byb2dpZBgCIAEo",
+            "CRIWCg5teGFjY2Vzc19jbHNpZBgDIAEoCRIzCg9yZWFkeV90aW1lc3RhbXAY",
+            "BCABKAsyGi5nb29nbGUucHJvdG9idWYuVGltZXN0YW1wIncKDVdvcmtlckNv",
+            "bW1hbmQSLwoHY29tbWFuZBgBIAEoCzIeLm14YWNjZXNzX2dhdGV3YXkudjEu",
+            "TXhDb21tYW5kEjUKEWVucXVldWVfdGltZXN0YW1wGAIgASgLMhouZ29vZ2xl",
+            "LnByb3RvYnVmLlRpbWVzdGFtcCKBAQoSV29ya2VyQ29tbWFuZFJlcGx5EjIK",
+            "BXJlcGx5GAEgASgLMiMubXhhY2Nlc3NfZ2F0ZXdheS52MS5NeENvbW1hbmRS",
+            "ZXBseRI3ChNjb21wbGV0ZWRfdGltZXN0YW1wGAIgASgLMhouZ29vZ2xlLnBy",
+            "b3RvYnVmLlRpbWVzdGFtcCIeCgxXb3JrZXJDYW5jZWwSDgoGcmVhc29uGAEg",
+            "ASgJIlEKDldvcmtlclNodXRkb3duEi8KDGdyYWNlX3BlcmlvZBgBIAEoCzIZ",
+            "Lmdvb2dsZS5wcm90b2J1Zi5EdXJhdGlvbhIOCgZyZWFzb24YAiABKAkiSAoR",
+            "V29ya2VyU2h1dGRvd25BY2sSMwoGc3RhdHVzGAEgASgLMiMubXhhY2Nlc3Nf",
+            "Z2F0ZXdheS52MS5Qcm90b2NvbFN0YXR1cyI6CgtXb3JrZXJFdmVudBIrCgVl",
+            "dmVudBgBIAEoCzIcLm14YWNjZXNzX2dhdGV3YXkudjEuTXhFdmVudCKlAgoP",
+            "V29ya2VySGVhcnRiZWF0EhkKEXdvcmtlcl9wcm9jZXNzX2lkGAEgASgFEi4K",
+            "BXN0YXRlGAIgASgOMh8ubXhhY2Nlc3Nfd29ya2VyLnYxLldvcmtlclN0YXRl",
+            "Ej8KG2xhc3Rfc3RhX2FjdGl2aXR5X3RpbWVzdGFtcBgDIAEoCzIaLmdvb2ds",
+            "ZS5wcm90b2J1Zi5UaW1lc3RhbXASHQoVcGVuZGluZ19jb21tYW5kX2NvdW50",
+            "GAQgASgNEiIKGm91dGJvdW5kX2V2ZW50X3F1ZXVlX2RlcHRoGAUgASgNEhsK",
+            "E2xhc3RfZXZlbnRfc2VxdWVuY2UYBiABKAQSJgoeY3VycmVudF9jb21tYW5k",
+            "X2NvcnJlbGF0aW9uX2lkGAcgASgJIvQBCgtXb3JrZXJGYXVsdBI5CghjYXRl",
+            "Z29yeRgBIAEoDjInLm14YWNjZXNzX3dvcmtlci52MS5Xb3JrZXJGYXVsdENh",
+            "dGVnb3J5EhYKDmNvbW1hbmRfbWV0aG9kGAIgASgJEhQKB2hyZXN1bHQYAyAB",
+            "KAVIAIgBARIWCg5leGNlcHRpb25fdHlwZRgEIAEoCRIaChJkaWFnbm9zdGlj",
+            "X21lc3NhZ2UYBSABKAkSPAoPcHJvdG9jb2xfc3RhdHVzGAYgASgLMiMubXhh",
+            "Y2Nlc3NfZ2F0ZXdheS52MS5Qcm90b2NvbFN0YXR1c0IKCghfaHJlc3VsdCqX",
+            "AgoLV29ya2VyU3RhdGUSHAoYV09SS0VSX1NUQVRFX1VOU1BFQ0lGSUVEEAAS",
+            "GQoVV09SS0VSX1NUQVRFX1NUQVJUSU5HEAESHAoYV09SS0VSX1NUQVRFX0hB",
+            "TkRTSEFLSU5HEAISIQodV09SS0VSX1NUQVRFX0lOSVRJQUxJWklOR19TVEEQ",
+            "AxIWChJXT1JLRVJfU1RBVEVfUkVBRFkQBBIiCh5XT1JLRVJfU1RBVEVfRVhF",
+            "Q1VUSU5HX0NPTU1BTkQQBRIeChpXT1JLRVJfU1RBVEVfU0hVVFRJTkdfRE9X",
+            "ThAGEhgKFFdPUktFUl9TVEFURV9TVE9QUEVEEAcSGAoUV09SS0VSX1NUQVRF",
+            "X0ZBVUxURUQQCCrHBAoTV29ya2VyRmF1bHRDYXRlZ29yeRIlCiFXT1JLRVJf",
+            "RkFVTFRfQ0FURUdPUllfVU5TUEVDSUZJRUQQABIrCidXT1JLRVJfRkFVTFRf",
+            "Q0FURUdPUllfSU5WQUxJRF9BUkdVTUVOVFMQARI3CjNXT1JLRVJfRkFVTFRf",
+            "Q0FURUdPUllfR0FURVdBWV9BVVRIRU5USUNBVElPTl9GQUlMRUQQAhIrCidX",
+            "T1JLRVJfRkFVTFRfQ0FURUdPUllfUFJPVE9DT0xfTUlTTUFUQ0gQAxIsCihX",
+            "T1JLRVJfRkFVTFRfQ0FURUdPUllfUFJPVE9DT0xfVklPTEFUSU9OEAQSKwon",
+            "V09SS0VSX0ZBVUxUX0NBVEVHT1JZX1BJUEVfRElTQ09OTkVDVEVEEAUSMgou",
+            "V09SS0VSX0ZBVUxUX0NBVEVHT1JZX01YQUNDRVNTX0NSRUFUSU9OX0ZBSUxF",
+            "RBAGEjEKLVdPUktFUl9GQVVMVF9DQVRFR09SWV9NWEFDQ0VTU19DT01NQU5E",
+            "X0ZBSUxFRBAHEjoKNldPUktFUl9GQVVMVF9DQVRFR09SWV9NWEFDQ0VTU19F",
+            "VkVOVF9DT05WRVJTSU9OX0ZBSUxFRBAIEiIKHldPUktFUl9GQVVMVF9DQVRF",
+            "R09SWV9TVEFfSFVORxAJEigKJFdPUktFUl9GQVVMVF9DQVRFR09SWV9RVUVV",
+            "RV9PVkVSRkxPVxAKEioKJldPUktFUl9GQVVMVF9DQVRFR09SWV9TSFVURE9X",
+            "Tl9USU1FT1VUEAtCJqoCI1pCLk1PTS5XVy5NeEdhdGV3YXkuQ29udHJhY3Rz",
+            "LlByb3RvYgZwcm90bzM="));
      descriptor = pbr::FileDescriptor.FromGeneratedCode(descriptorData,
          new pbr::FileDescriptor[] { global::Google.Protobuf.WellKnownTypes.DurationReflection.Descriptor, global::Google.Protobuf.WellKnownTypes.TimestampReflection.Descriptor, global::ZB.MOM.WW.MxGateway.Contracts.Proto.MxaccessGatewayReflection.Descriptor, },
          new pbr::GeneratedClrTypeInfo(new[] {typeof(global::ZB.MOM.WW.MxGateway.Contracts.Proto.WorkerState), typeof(global::ZB.MOM.WW.MxGateway.Contracts.Proto.WorkerFaultCategory), }, null, new pbr::GeneratedClrTypeInfo[] {
            new pbr::GeneratedClrTypeInfo(typeof(global::ZB.MOM.WW.MxGateway.Contracts.Proto.WorkerEnvelope), global::ZB.MOM.WW.MxGateway.Contracts.Proto.WorkerEnvelope.Parser, new[]{ "ProtocolVersion", "SessionId", "Sequence", "CorrelationId", "GatewayHello", "WorkerHello", "WorkerReady", "WorkerCommand", "WorkerCommandReply", "WorkerCancel", "WorkerShutdown", "WorkerShutdownAck", "WorkerEvent", "WorkerHeartbeat", "WorkerFault" }, new[]{ "Body" }, null, null, null),
-            new pbr::GeneratedClrTypeInfo(typeof(global::ZB.MOM.WW.MxGateway.Contracts.Proto.GatewayHello), global::ZB.MOM.WW.MxGateway.Contracts.Proto.GatewayHello.Parser, new[]{ "SupportedProtocolVersion", "Nonce", "GatewayVersion" }, null, null, null, null),
+            new pbr::GeneratedClrTypeInfo(typeof(global::ZB.MOM.WW.MxGateway.Contracts.Proto.GatewayHello), global::ZB.MOM.WW.MxGateway.Contracts.Proto.GatewayHello.Parser, new[]{ "SupportedProtocolVersion", "Nonce", "GatewayVersion", "MaxFrameBytes" }, null, null, null, null),
            new pbr::GeneratedClrTypeInfo(typeof(global::ZB.MOM.WW.MxGateway.Contracts.Proto.WorkerHello), global::ZB.MOM.WW.MxGateway.Contracts.Proto.WorkerHello.Parser, new[]{ "ProtocolVersion", "Nonce", "WorkerProcessId", "WorkerVersion" }, null, null, null, null),
            new pbr::GeneratedClrTypeInfo(typeof(global::ZB.MOM.WW.MxGateway.Contracts.Proto.WorkerReady), global::ZB.MOM.WW.MxGateway.Contracts.Proto.WorkerReady.Parser, new[]{ "WorkerProcessId", "MxaccessProgid", "MxaccessClsid", "ReadyTimestamp" }, null, null, null, null),
            new pbr::GeneratedClrTypeInfo(typeof(global::ZB.MOM.WW.MxGateway.Contracts.Proto.WorkerCommand), global::ZB.MOM.WW.MxGateway.Contracts.Proto.WorkerCommand.Parser, new[]{ "Command", "EnqueueTimestamp" }, null, null, null, null),
@@ -1108,6 +1109,7 @@ namespace ZB.MOM.WW.MxGateway.Contracts.Proto {
      supportedProtocolVersion_ = other.supportedProtocolVersion_;
      nonce_ = other.nonce_;
      gatewayVersion_ = other.gatewayVersion_;
+      maxFrameBytes_ = other.maxFrameBytes_;
      _unknownFields = pb::UnknownFieldSet.Clone(other._unknownFields);
    }

@@ -1153,6 +1155,25 @@ namespace ZB.MOM.WW.MxGateway.Contracts.Proto {
      }
    }

+    /// <summary>Field number for the "max_frame_bytes" field.</summary>
+    public const int MaxFrameBytesFieldNumber = 4;
+    private uint maxFrameBytes_;
+    /// <summary>
+    /// Maximum worker-frame payload size, in bytes, negotiated by the gateway from its
+    /// configured pipe limit. The worker adopts this as its frame-protocol MaxMessageBytes
+    /// instead of a hard-coded default; 0 (an older gateway that never set the field) means
+    /// "use the worker's built-in default". Sits above the public gRPC cap by an
+    /// envelope-overhead margin so an accepted gRPC payload always fits one worker frame.
+    /// </summary>
+    [global::System.Diagnostics.DebuggerNonUserCodeAttribute]
+    [global::System.CodeDom.Compiler.GeneratedCode("protoc", null)]
+    public uint MaxFrameBytes {
+      get { return maxFrameBytes_; }
+      set {
+        maxFrameBytes_ = value;
+      }
+    }
+
    [global::System.Diagnostics.DebuggerNonUserCodeAttribute]
    [global::System.CodeDom.Compiler.GeneratedCode("protoc", null)]
    public override bool Equals(object other) {
@@ -1171,6 +1192,7 @@ namespace ZB.MOM.WW.MxGateway.Contracts.Proto {
      if (SupportedProtocolVersion != other.SupportedProtocolVersion) return false;
      if (Nonce != other.Nonce) return false;
      if (GatewayVersion != other.GatewayVersion) return false;
+      if (MaxFrameBytes != other.MaxFrameBytes) return false;
      return Equals(_unknownFields, other._unknownFields);
    }

@@ -1181,6 +1203,7 @@ namespace ZB.MOM.WW.MxGateway.Contracts.Proto {
      if (SupportedProtocolVersion != 0) hash ^= SupportedProtocolVersion.GetHashCode();
      if (Nonce.Length != 0) hash ^= Nonce.GetHashCode();
      if (GatewayVersion.Length != 0) hash ^= GatewayVersion.GetHashCode();
+      if (MaxFrameBytes != 0) hash ^= MaxFrameBytes.GetHashCode();
      if (_unknownFields != null) {
        hash ^= _unknownFields.GetHashCode();
      }
@@ -1211,6 +1234,10 @@ namespace ZB.MOM.WW.MxGateway.Contracts.Proto {
        output.WriteRawTag(26);
        output.WriteString(GatewayVersion);
      }
+      if (MaxFrameBytes != 0) {
+        output.WriteRawTag(32);
+        output.WriteUInt32(MaxFrameBytes);
+      }
      if (_unknownFields != null) {
        _unknownFields.WriteTo(output);
      }
@@ -1233,6 +1260,10 @@ namespace ZB.MOM.WW.MxGateway.Contracts.Proto {
        output.WriteRawTag(26);
        output.WriteString(GatewayVersion);
      }
+      if (MaxFrameBytes != 0) {
+        output.WriteRawTag(32);
+        output.WriteUInt32(MaxFrameBytes);
+      }
      if (_unknownFields != null) {
        _unknownFields.WriteTo(ref output);
      }
@@ -1252,6 +1283,9 @@ namespace ZB.MOM.WW.MxGateway.Contracts.Proto {
      if (GatewayVersion.Length != 0) {
        size += 1 + pb::CodedOutputStream.ComputeStringSize(GatewayVersion);
      }
+      if (MaxFrameBytes != 0) {
+        size += 1 + pb::CodedOutputStream.ComputeUInt32Size(MaxFrameBytes);
+      }
      if (_unknownFields != null) {
        size += _unknownFields.CalculateSize();
      }
@@ -1273,6 +1307,9 @@ namespace ZB.MOM.WW.MxGateway.Contracts.Proto {
      if (other.GatewayVersion.Length != 0) {
        GatewayVersion = other.GatewayVersion;
      }
+      if (other.MaxFrameBytes != 0) {
+        MaxFrameBytes = other.MaxFrameBytes;
+      }
      _unknownFields = pb::UnknownFieldSet.MergeFrom(_unknownFields, other._unknownFields);
    }

@@ -1304,6 +1341,10 @@ namespace ZB.MOM.WW.MxGateway.Contracts.Proto {
            GatewayVersion = input.ReadString();
            break;
          }
+          case 32: {
+            MaxFrameBytes = input.ReadUInt32();
+            break;
+          }
        }
      }
    #endif
@@ -1335,6 +1376,10 @@ namespace ZB.MOM.WW.MxGateway.Contracts.Proto {
            GatewayVersion = input.ReadString();
            break;
          }
+          case 32: {
+            MaxFrameBytes = input.ReadUInt32();
+            break;
+          }
        }
      }
    }
@@ -42,6 +42,12 @@ message GatewayHello {
  uint32 supported_protocol_version = 1;
  string nonce = 2;
  string gateway_version = 3;
+  // Maximum worker-frame payload size, in bytes, negotiated by the gateway from its
+  // configured pipe limit. The worker adopts this as its frame-protocol MaxMessageBytes
+  // instead of a hard-coded default; 0 (an older gateway that never set the field) means
+  // "use the worker's built-in default". Sits above the public gRPC cap by an
+  // envelope-overhead margin so an accepted gRPC payload always fits one worker frame.
+  uint32 max_frame_bytes = 4;
 }

 message WorkerHello {
@@ -1,6 +1,7 @@
 using ZB.MOM.WW.Auth.Abstractions.Ldap;
 using ZB.MOM.WW.Configuration;
 using ZB.MOM.WW.MxGateway.Contracts;
+using ZB.MOM.WW.MxGateway.Server.Workers;

 namespace ZB.MOM.WW.MxGateway.Server.Configuration;

@@ -46,6 +47,7 @@ public sealed class GatewayOptionsValidator : OptionsValidatorBase<GatewayOption
        ValidateEvents(options.Events, builder);
        ValidateDashboard(options.Dashboard, builder, _isProduction);
        ValidateProtocol(options.Protocol, builder);
+        ValidateFrameSizeHeadroom(options.Worker, options.Protocol, builder);
        ValidateAlarms(options.Alarms, builder);
        ValidateTls(options.Tls, builder);
        ValidateSecurity(options.Security, builder);
@@ -466,6 +468,35 @@ public sealed class GatewayOptionsValidator : OptionsValidatorBase<GatewayOption
        }
    }

+    // The worker-frame (pipe) maximum must stay above the public gRPC cap by the envelope-overhead
+    // reserve, otherwise a maximally-sized accepted gRPC payload does not fit one worker frame once
+    // wrapped in a WorkerEnvelope and the outbound write faults the whole session (IPC-03). Fail fast
+    // at startup rather than mid-traffic. Only checked when both knobs are themselves in range so the
+    // message is not doubled up with the individual range errors.
+    private static void ValidateFrameSizeHeadroom(
+        WorkerOptions worker,
+        ProtocolOptions protocol,
+        ValidationBuilder builder)
+    {
+        bool workerInRange = worker.MaxMessageBytes is >= MinimumMaxMessageBytes and <= MaximumMaxMessageBytes;
+        bool grpcInRange = protocol.MaxGrpcMessageBytes is >= MinimumMaxMessageBytes and <= MaximumMaxMessageBytes;
+        if (!workerInRange || !grpcInRange)
+        {
+            return;
+        }
+
+        long requiredWorkerMax =
+            (long)protocol.MaxGrpcMessageBytes + WorkerFrameProtocolOptions.EnvelopeOverheadReserveBytes;
+        if (worker.MaxMessageBytes < requiredWorkerMax)
+        {
+            builder.Add(
+                $"MxGateway:Worker:MaxMessageBytes ({worker.MaxMessageBytes}) must be at least "
+                + $"MxGateway:Protocol:MaxGrpcMessageBytes ({protocol.MaxGrpcMessageBytes}) plus the "
+                + $"{WorkerFrameProtocolOptions.EnvelopeOverheadReserveBytes}-byte worker-frame envelope reserve "
+                + $"(>= {requiredWorkerMax}).");
+        }
+    }
+
    private static void AddIfBlank(string? value, string message, ValidationBuilder builder)
    {
        builder.RequireThat(!string.IsNullOrWhiteSpace(value), message);
@@ -33,6 +33,14 @@ public sealed class WorkerOptions
    /// <summary>The grace period in seconds after a heartbeat before considering the worker unresponsive.</summary>
    public int HeartbeatGraceSeconds { get; init; } = 15;

-    /// <summary>The maximum message size in bytes for IPC communication.</summary>
-    public int MaxMessageBytes { get; init; } = 16 * 1024 * 1024;
+    /// <summary>
+    /// The maximum worker-frame (pipe) message size in bytes. Must stay at least
+    /// <see cref="Workers.WorkerFrameProtocolOptions.EnvelopeOverheadReserveBytes"/> above
+    /// <see cref="ProtocolOptions.MaxGrpcMessageBytes"/> so a maximally-sized accepted gRPC payload
+    /// still fits one worker frame (IPC-03); the gateway conveys this value to the worker in the
+    /// handshake (<c>GatewayHello.max_frame_bytes</c>, IPC-02). Default is the 16 MB public gRPC cap
+    /// plus that reserve.
+    /// </summary>
+    public int MaxMessageBytes { get; init; } =
+        (16 * 1024 * 1024) + Workers.WorkerFrameProtocolOptions.EnvelopeOverheadReserveBytes;
 }
@@ -11,6 +11,18 @@ namespace ZB.MOM.WW.MxGateway.Server.Dashboard;
 /// (b) authentication is fully disabled, or (c) the request is from loopback
 /// and <c>MxGateway:Dashboard:AllowAnonymousLocalhost</c> is on.
 /// </summary>
+/// <remarks>
+/// The environment bypasses in (b) and (c) grant <em>read-only</em> access only: they
+/// satisfy a requirement that includes <see cref="DashboardRoles.Viewer"/> (i.e.
+/// <see cref="DashboardAuthorizationRequirement.AnyDashboardRole"/>) but never the
+/// <see cref="DashboardAuthorizationRequirement.AdminOnly"/> requirement. Destructive/admin
+/// surfaces still require a real Admin role claim, preserving the documented
+/// "anonymous localhost is read-only" contract at the policy layer rather than by accident
+/// in downstream service re-checks. The loopback test trusts
+/// <c>Connection.RemoteIpAddress</c>; if forwarded-headers middleware is ever added upstream,
+/// <see cref="IsLoopbackRequest"/> must be revisited so a spoofed <c>X-Forwarded-For</c>
+/// cannot masquerade as loopback.
+/// </remarks>
 public sealed class DashboardAuthorizationHandler(
    IHttpContextAccessor httpContextAccessor,
    IOptions<GatewayOptions> options) : AuthorizationHandler<DashboardAuthorizationRequirement>
@@ -22,14 +34,18 @@ public sealed class DashboardAuthorizationHandler(
    {
        GatewayOptions gatewayOptions = options.Value;

-        if (gatewayOptions.Authentication.Mode == AuthenticationMode.Disabled)
+        // Environment bypasses are read-only: they satisfy the Viewer requirement
+        // (AnyDashboardRole) but never AdminOnly, which lacks Viewer in RequiredRoles.
+        bool grantsReadOnly = requirement.RequiredRoles.Contains(DashboardRoles.Viewer);
+
+        if (grantsReadOnly && gatewayOptions.Authentication.Mode == AuthenticationMode.Disabled)
        {
            context.Succeed(requirement);

            return Task.CompletedTask;
        }

-        if (gatewayOptions.Dashboard.AllowAnonymousLocalhost && IsLoopbackRequest())
+        if (grantsReadOnly && gatewayOptions.Dashboard.AllowAnonymousLocalhost && IsLoopbackRequest())
        {
            context.Succeed(requirement);

@@ -1,6 +1,8 @@
 using System.Security.Claims;
+using System.Text.Json;
 using Microsoft.Extensions.Logging;
 using Microsoft.Extensions.Logging.Abstractions;
+using ZB.MOM.WW.Audit;
 using ZB.MOM.WW.MxGateway.Server.Sessions;

 namespace ZB.MOM.WW.MxGateway.Server.Dashboard;
@@ -8,21 +10,35 @@ namespace ZB.MOM.WW.MxGateway.Server.Dashboard;
 /// <summary>
 /// Default implementation of <see cref="IDashboardSessionAdminService"/>: gates
 /// destructive session actions on the <see cref="DashboardRoles.Admin"/> role,
-/// audit-logs successful operations, and converts <see cref="SessionManagerException"/>
-/// (and any other unexpected exceptions) into <see cref="DashboardSessionAdminResult.Fail(string)"/>
-/// so the Blazor pages never see a raw exception.
+/// records each attempt as a canonical <see cref="AuditEvent"/> through <see cref="IAuditWriter"/>
+/// (in addition to the operational <see cref="ILogger"/> line), and converts
+/// <see cref="SessionManagerException"/> (and any other unexpected exceptions) into
+/// <see cref="DashboardSessionAdminResult.Fail(string)"/> so the Blazor pages never see a raw exception.
 /// </summary>
 /// <remarks>
 ///     The constant <c>dashboard-admin-kill</c> is the reason passed to
 ///     <see cref="ISessionManager.KillWorkerAsync"/> and forwarded as the
 ///     <c>reason</c> tag on the <c>mxgateway.workers.killed</c> counter and in
-///     the worker-kill audit log entries.
+///     the worker-kill audit log entries. Destructive dashboard actions write durable,
+///     queryable audit rows (actions <c>dashboard-close-session</c> / <c>dashboard-kill-worker</c>)
+///     to the canonical <c>audit_event</c> store, mirroring the API-key management path so a
+///     worker killed mid-production is not visible only in a rotatable <see cref="ILogger"/> line.
 /// </remarks>
 public sealed class DashboardSessionAdminService(
    ISessionManager sessionManager,
    IHttpContextAccessor httpContextAccessor,
+    IAuditWriter auditWriter,
    ILogger<DashboardSessionAdminService>? logger = null) : IDashboardSessionAdminService
 {
+    /// <summary>Canonical <see cref="AuditEvent.Category"/> for dashboard session-admin actions.</summary>
+    internal const string SessionAdminCategory = "SessionAdmin";
+
+    /// <summary>Canonical <see cref="AuditEvent.Action"/> for a dashboard session close.</summary>
+    internal const string CloseSessionAction = "dashboard-close-session";
+
+    /// <summary>Canonical <see cref="AuditEvent.Action"/> for a dashboard worker kill.</summary>
+    internal const string KillWorkerAction = "dashboard-kill-worker";
+
    private const string UnauthorizedMessage = "Sign in as an Admin to close sessions or kill workers.";
    private const string KillReason = "dashboard-admin-kill";

@@ -46,6 +62,8 @@ public sealed class DashboardSessionAdminService(
    {
        if (!CanManage(user))
        {
+            await WriteAuditAsync(user, CloseSessionAction, sessionId, AuditOutcome.Denied, "unauthorized", cancellationToken)
+                .ConfigureAwait(false);
            return DashboardSessionAdminResult.Fail(UnauthorizedMessage);
        }

@@ -68,6 +86,15 @@ public sealed class DashboardSessionAdminService(
                ResolveRemoteAddress(),
                result.AlreadyClosed);

+            await WriteAuditAsync(
+                    user,
+                    CloseSessionAction,
+                    sessionId,
+                    AuditOutcome.Success,
+                    result.AlreadyClosed ? "alreadyClosed" : "closed",
+                    cancellationToken)
+                .ConfigureAwait(false);
+
            return DashboardSessionAdminResult.Success(
                result.AlreadyClosed
                    ? $"Session {sessionId} was already closed."
@@ -75,6 +102,8 @@ public sealed class DashboardSessionAdminService(
        }
        catch (SessionManagerException exception) when (exception.ErrorCode == SessionManagerErrorCode.SessionNotFound)
        {
+            await WriteAuditAsync(user, CloseSessionAction, sessionId, AuditOutcome.Failure, "not-found", cancellationToken)
+                .ConfigureAwait(false);
            return DashboardSessionAdminResult.Fail($"Session {sessionId} was not found.");
        }
        catch (SessionManagerException exception)
@@ -84,6 +113,8 @@ public sealed class DashboardSessionAdminService(
                "Dashboard admin {Actor} close failed for session {SessionId}.",
                actor,
                sessionId);
+            await WriteAuditAsync(user, CloseSessionAction, sessionId, AuditOutcome.Failure, exception.Message, cancellationToken)
+                .ConfigureAwait(false);
            return DashboardSessionAdminResult.Fail(
                $"Close failed: {exception.Message}");
        }
@@ -98,6 +129,8 @@ public sealed class DashboardSessionAdminService(
                "Dashboard admin {Actor} close failed unexpectedly for session {SessionId}.",
                actor,
                sessionId);
+            await WriteAuditAsync(user, CloseSessionAction, sessionId, AuditOutcome.Failure, "unexpected", cancellationToken)
+                .ConfigureAwait(false);
            return DashboardSessionAdminResult.Fail(
                $"Close failed unexpectedly for session {sessionId}. See the gateway log for details.");
        }
@@ -111,6 +144,8 @@ public sealed class DashboardSessionAdminService(
    {
        if (!CanManage(user))
        {
+            await WriteAuditAsync(user, KillWorkerAction, sessionId, AuditOutcome.Denied, "unauthorized", cancellationToken)
+                .ConfigureAwait(false);
            return DashboardSessionAdminResult.Fail(UnauthorizedMessage);
        }

@@ -133,6 +168,15 @@ public sealed class DashboardSessionAdminService(
                ResolveRemoteAddress(),
                result.AlreadyClosed);

+            await WriteAuditAsync(
+                    user,
+                    KillWorkerAction,
+                    sessionId,
+                    AuditOutcome.Success,
+                    result.AlreadyClosed ? "alreadyClosed" : "killed",
+                    cancellationToken)
+                .ConfigureAwait(false);
+
            return DashboardSessionAdminResult.Success(
                result.AlreadyClosed
                    ? $"Session {sessionId} was already closed."
@@ -140,6 +184,8 @@ public sealed class DashboardSessionAdminService(
        }
        catch (SessionManagerException exception) when (exception.ErrorCode == SessionManagerErrorCode.SessionNotFound)
        {
+            await WriteAuditAsync(user, KillWorkerAction, sessionId, AuditOutcome.Failure, "not-found", cancellationToken)
+                .ConfigureAwait(false);
            return DashboardSessionAdminResult.Fail($"Session {sessionId} was not found.");
        }
        catch (SessionManagerException exception)
@@ -149,6 +195,8 @@ public sealed class DashboardSessionAdminService(
                "Dashboard admin {Actor} kill failed for session {SessionId}.",
                actor,
                sessionId);
+            await WriteAuditAsync(user, KillWorkerAction, sessionId, AuditOutcome.Failure, exception.Message, cancellationToken)
+                .ConfigureAwait(false);
            return DashboardSessionAdminResult.Fail(
                $"Kill failed: {exception.Message}");
        }
@@ -163,6 +211,8 @@ public sealed class DashboardSessionAdminService(
                "Dashboard admin {Actor} kill failed unexpectedly for session {SessionId}.",
                actor,
                sessionId);
+            await WriteAuditAsync(user, KillWorkerAction, sessionId, AuditOutcome.Failure, "unexpected", cancellationToken)
+                .ConfigureAwait(false);
            return DashboardSessionAdminResult.Fail(
                $"Kill failed unexpectedly for session {sessionId}. See the gateway log for details.");
        }
@@ -177,4 +227,51 @@ public sealed class DashboardSessionAdminService(
    {
        return httpContextAccessor.HttpContext?.Connection.RemoteIpAddress?.ToString();
    }
+
+    /// <summary>
+    /// Writes a canonical <see cref="AuditEvent"/> for a dashboard session-admin action through the
+    /// best-effort <see cref="IAuditWriter"/> (failures are swallowed/logged by the writer, so this
+    /// never throws and never masks the operation result). <see cref="AuditEvent.Actor"/> is the LDAP
+    /// operator, <see cref="AuditEvent.Target"/> the session id, and <paramref name="detail"/> is wrapped
+    /// as the <c>detail</c> field of the JSON extension bag.
+    /// </summary>
+    private async Task WriteAuditAsync(
+        ClaimsPrincipal user,
+        string action,
+        string sessionId,
+        AuditOutcome outcome,
+        string? detail,
+        CancellationToken cancellationToken)
+    {
+        AuditEvent auditEvent = new()
+        {
+            EventId = Guid.NewGuid(),
+            OccurredAtUtc = DateTimeOffset.UtcNow,
+            Actor = ResolveActor(user),
+            Action = action,
+            Outcome = outcome,
+            Category = SessionAdminCategory,
+            Target = sessionId,
+            SourceNode = ResolveRemoteAddress(),
+            CorrelationId = ParseCorrelationId(),
+            DetailsJson = WrapDetail(detail),
+        };
+
+        await auditWriter.WriteAsync(auditEvent, cancellationToken).ConfigureAwait(false);
+    }
+
+    /// <summary>
+    /// Derives a correlation id from the request trace identifier when it is a well-formed GUID;
+    /// otherwise null (the default <c>HttpContext.TraceIdentifier</c> is the connection:request form,
+    /// not a GUID, so it correlates to null rather than fabricating one).
+    /// </summary>
+    private Guid? ParseCorrelationId() =>
+        Guid.TryParse(httpContextAccessor.HttpContext?.TraceIdentifier, out Guid correlationId)
+            ? correlationId
+            : null;
+
+    private static string? WrapDetail(string? detail) =>
+        detail is null
+            ? null
+            : JsonSerializer.Serialize(new Dictionary<string, string> { ["detail"] = detail });
 }
@@ -946,6 +946,7 @@ public sealed class MxAccessGatewayService(
            WorkerClientErrorCode.GatewayShutdown => StatusCode.Cancelled,
            WorkerClientErrorCode.InvalidState => StatusCode.FailedPrecondition,
            WorkerClientErrorCode.ProtocolViolation => StatusCode.Internal,
+            WorkerClientErrorCode.CommandTooLarge => StatusCode.ResourceExhausted,
            _ => StatusCode.Unavailable,
        };

@@ -5,6 +5,13 @@ namespace ZB.MOM.WW.MxGateway.Server.Grpc;

 public sealed class MxAccessGrpcRequestValidator
 {
+    // Upper bound on a single DrainEvents request. DrainEvents is a diagnostics RPC that returns
+    // buffered events in one non-streaming reply, so an unbounded max_events could pack the whole
+    // queue into a session-killing frame (IPC-04). The worker independently caps each reply at its
+    // own MaxDrainEventsPerReply; this public bound rejects an obviously-abusive request loudly at
+    // the boundary. max_events = 0 is allowed and means "the worker's default batch cap".
+    private const uint MaxDrainEventsPerRequest = 10_000;
+
    /// <summary>Validates an open session request.</summary>
    /// <param name="request">The request to validate.</param>
    public void ValidateOpenSession(OpenSessionRequest request)
@@ -69,6 +76,14 @@ public sealed class MxAccessGrpcRequestValidator
            throw InvalidArgument(
                $"Command kind {command.Kind} requires payload {expectedPayload} but received {command.PayloadCase}.");
        }
+
+        // The payload case now matches the kind, so command.DrainEvents is non-null here.
+        if (command.Kind is MxCommandKind.DrainEvents && command.DrainEvents.MaxEvents > MaxDrainEventsPerRequest)
+        {
+            throw InvalidArgument(
+                $"DrainEvents max_events ({command.DrainEvents.MaxEvents}) must not exceed {MaxDrainEventsPerRequest}; "
+                + "use 0 to request the worker default batch cap.");
+        }
    }

    private static MxCommand.PayloadOneofCase ExpectedPayload(MxCommandKind kind)
@@ -351,7 +351,11 @@ public sealed class GatewayMetrics : IDisposable
            _heartbeatFailures++;
        }

-        _heartbeatFailuresCounter.Add(1, new KeyValuePair<string, object?>("session_id", sessionId));
+        // Exported without a session_id tag: per-session attribution is unbounded cardinality
+        // (every session ever opened mints a new exporter time series, bloating Prometheus/OTLP
+        // storage on long-running gateways). Per-session detail stays available via the dashboard
+        // snapshot and the log scope; the exported counter tracks only the aggregate.
+        _heartbeatFailuresCounter.Add(1);
    }

    /// <summary>
@@ -163,7 +163,15 @@ public sealed class SelfSignedCertificateProvider
            // temp file empty, harden its permissions, and only then write the PFX into
            // the already-protected file. The temp path is in the same directory as the
            // target so the Move is atomic and preserves the hardened DACL/mode.
-            string temp = path + ".tmp";
+            //
+            // The temp name carries a unique suffix rather than a fixed "<path>.tmp": two
+            // processes (or two parallel callers) generating to the same target must not
+            // collide on one temp file. On Windows a fixed name makes the second writer's
+            // File.Create/Move fail with "the process cannot access the file ... because it
+            // is being used by another process"; a unique name lets each generation stage
+            // independently, and the final atomic Move (last-writer-wins) still yields a
+            // valid, equivalent certificate at the shared path.
+            string temp = $"{path}.{Guid.NewGuid():N}.tmp";
            using (File.Create(temp)) { }
            HardenPermissions(temp);

@@ -24,6 +24,13 @@ public sealed class WorkerClient : IWorkerClient
    private readonly WorkerFrameWriter _writer;
    private readonly Channel<WorkerEnvelope> _outboundEnvelopes;
    private readonly Channel<WorkerEvent> _events;
+
+    // Staging hand-off between the read loop and the dedicated event writer. The read loop writes
+    // here with a non-blocking TryWrite so a full consumer channel (_events) can never stall the read
+    // loop behind an event — replies and heartbeats keep flowing (GWC-04). Unbounded, but only fills
+    // during the bounded EventChannelFullModeTimeout window before EventWriteLoopAsync faults on a
+    // sustained backlog, after which the read loop stops.
+    private readonly Channel<WorkerEvent> _eventStaging;
    private readonly ConcurrentDictionary<string, PendingCommand> _pendingCommands = new(StringComparer.Ordinal);
    private readonly SemaphoreSlim _pendingCommandSlots;
    private readonly CancellationTokenSource _stopCts = new();
@@ -35,6 +42,7 @@ public sealed class WorkerClient : IWorkerClient
    private int _eventsReaderClaimed;
    private Task? _readLoopTask;
    private Task? _writeLoopTask;
+    private Task? _eventWriteLoopTask;
    private Task? _heartbeatLoopTask;
    private bool _workerStartRecorded;
    private bool _disposed;
@@ -82,6 +90,14 @@ public sealed class WorkerClient : IWorkerClient
                FullMode = BoundedChannelFullMode.Wait,
                AllowSynchronousContinuations = false,
            });
+        _eventStaging = Channel.CreateUnbounded<WorkerEvent>(
+            new UnboundedChannelOptions
+            {
+                // The read loop is the only writer; EventWriteLoopAsync is the only reader.
+                SingleReader = true,
+                SingleWriter = true,
+                AllowSynchronousContinuations = false,
+            });
        _lastHeartbeatAt = _timeProvider.GetUtcNow();
    }

@@ -144,6 +160,7 @@ public sealed class WorkerClient : IWorkerClient
        MarkReady(readyEnvelope.WorkerReady);

        _readLoopTask = Task.Run(ReadLoopAsync);
+        _eventWriteLoopTask = Task.Run(EventWriteLoopAsync);
        _heartbeatLoopTask = Task.Run(HeartbeatLoopAsync);
    }

@@ -187,7 +204,23 @@ public sealed class WorkerClient : IWorkerClient

        try
        {
-            await EnqueueAsync(CreateCommandEnvelope(correlationId, command), cancellationToken).ConfigureAwait(false);
+            WorkerEnvelope commandEnvelope = CreateCommandEnvelope(correlationId, command);
+
+            // Reject an oversized command at the enqueue boundary so only this correlation fails
+            // (ResourceExhausted) rather than the frame reaching the write loop and faulting the whole
+            // session (IPC-03). Command envelopes are the only gateway-authored outbound payload whose
+            // size the caller controls; checking here keeps a MessageTooLarge in the write loop a
+            // genuine desync signal.
+            int envelopeSize = commandEnvelope.CalculateSize();
+            if (envelopeSize > _connection.FrameOptions.MaxMessageBytes)
+            {
+                throw new WorkerClientException(
+                    WorkerClientErrorCode.CommandTooLarge,
+                    $"Worker command {method} serializes to {envelopeSize} bytes, exceeding the negotiated "
+                    + $"worker-frame maximum of {_connection.FrameOptions.MaxMessageBytes} bytes.");
+            }
+
+            await EnqueueAsync(commandEnvelope, cancellationToken).ConfigureAwait(false);
            using CancellationTokenSource timeoutCts = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
            Task timeoutTask = Task.Delay(timeout, timeoutCts.Token);
            Task<WorkerCommandReply> replyTask = pendingCommand.Task;
@@ -328,6 +361,7 @@ public sealed class WorkerClient : IWorkerClient
        KillOwnedProcess("Dispose");
        _stopCts.Cancel();
        _outboundEnvelopes.Writer.TryComplete();
+        _eventStaging.Writer.TryComplete();
        _events.Writer.TryComplete();
        CompletePendingCommands(
            new WorkerClientException(
@@ -382,7 +416,7 @@ public sealed class WorkerClient : IWorkerClient
            while (!_stopCts.IsCancellationRequested)
            {
                WorkerEnvelope envelope = await _reader.ReadAsync(_stopCts.Token).ConfigureAwait(false);
-                await DispatchEnvelopeAsync(envelope, _stopCts.Token).ConfigureAwait(false);
+                DispatchEnvelope(envelope);
            }
        }
        catch (OperationCanceledException) when (_stopCts.IsCancellationRequested || IsTerminalState())
@@ -481,12 +515,14 @@ public sealed class WorkerClient : IWorkerClient
        return true;
    }

-    /// <summary>Routes received envelope to appropriate handler.</summary>
+    /// <summary>
+    ///     Routes a received envelope to its handler. Every branch dispatches synchronously and
+    ///     immediately — the event branch only stages the event for the dedicated writer — so a full
+    ///     event channel can never delay a command reply, heartbeat, fault, or shutdown ack behind an
+    ///     event backlog (GWC-04).
+    /// </summary>
    /// <param name="envelope">The envelope to dispatch.</param>
-    /// <param name="cancellationToken">Cancellation token.</param>
-    private async Task DispatchEnvelopeAsync(
-        WorkerEnvelope envelope,
-        CancellationToken cancellationToken)
+    private void DispatchEnvelope(WorkerEnvelope envelope)
    {
        switch (envelope.BodyCase)
        {
@@ -494,7 +530,7 @@ public sealed class WorkerClient : IWorkerClient
                CompleteCommand(envelope);
                break;
            case WorkerEnvelope.BodyOneofCase.WorkerEvent:
-                await EnqueueWorkerEventAsync(envelope.WorkerEvent, cancellationToken).ConfigureAwait(false);
+                StageWorkerEvent(envelope.WorkerEvent);
                break;
            case WorkerEnvelope.BodyOneofCase.WorkerHeartbeat:
                MarkHeartbeat(envelope.WorkerHeartbeat);
@@ -517,6 +553,45 @@ public sealed class WorkerClient : IWorkerClient
        }
    }

+    /// <summary>
+    ///     Hands a received worker event to the dedicated event writer without blocking the read loop.
+    ///     The staging channel is unbounded and this is the only writer, so <c>TryWrite</c> always
+    ///     succeeds unless the channel has been completed during shutdown — in which case the event is
+    ///     safely dropped because the client is closing. Backpressure and the sustained-overflow fault
+    ///     are applied by <see cref="EventWriteLoopAsync"/> against the bounded consumer channel,
+    ///     off the read loop (GWC-04).
+    /// </summary>
+    /// <param name="workerEvent">The event received from the worker.</param>
+    private void StageWorkerEvent(WorkerEvent workerEvent)
+    {
+        if (workerEvent.Event is not null)
+        {
+            _metrics?.EventReceived(SessionId, workerEvent.Event.Family.ToString());
+        }
+
+        _eventStaging.Writer.TryWrite(workerEvent);
+    }
+
+    /// <summary>
+    ///     Drains staged worker events and applies the bounded-channel backpressure (and
+    ///     sustained-overflow fault) on a dedicated task, so the timed <see cref="Channel"/> write
+    ///     never runs on the read loop (GWC-04). Mirrors <see cref="WriteLoopAsync"/> for events.
+    /// </summary>
+    private async Task EventWriteLoopAsync()
+    {
+        try
+        {
+            await foreach (WorkerEvent workerEvent in
+                _eventStaging.Reader.ReadAllAsync(_stopCts.Token).ConfigureAwait(false))
+            {
+                await EnqueueWorkerEventAsync(workerEvent, _stopCts.Token).ConfigureAwait(false);
+            }
+        }
+        catch (OperationCanceledException) when (_stopCts.IsCancellationRequested || IsTerminalState())
+        {
+        }
+    }
+
    /// <summary>
    ///     Enqueues a worker event for client consumption. The channel is
    ///     configured with <see cref="BoundedChannelFullMode.Wait"/>
@@ -536,11 +611,6 @@ public sealed class WorkerClient : IWorkerClient
        WorkerEvent workerEvent,
        CancellationToken cancellationToken)
    {
-        if (workerEvent.Event is not null)
-        {
-            _metrics?.EventReceived(SessionId, workerEvent.Event.Family.ToString());
-        }
-
        if (_events.Writer.TryWrite(workerEvent))
        {
            int queueDepth = Interlocked.Increment(ref _eventQueueDepth);
@@ -731,6 +801,7 @@ public sealed class WorkerClient : IWorkerClient

        _stopCts.Cancel();
        _outboundEnvelopes.Writer.TryComplete();
+        _eventStaging.Writer.TryComplete();
        _events.Writer.TryComplete();
        CompletePendingCommands(
            new WorkerClientException(
@@ -764,6 +835,7 @@ public sealed class WorkerClient : IWorkerClient

        _stopCts.Cancel();
        _outboundEnvelopes.Writer.TryComplete(fault);
+        _eventStaging.Writer.TryComplete(fault);
        _events.Writer.TryComplete(fault);
        KillOwnedProcess(errorCode.ToString());
        CompletePendingCommands(fault);
@@ -910,6 +982,10 @@ public sealed class WorkerClient : IWorkerClient
                SupportedProtocolVersion = _connection.FrameOptions.ProtocolVersion,
                Nonce = _connection.Nonce,
                GatewayVersion = typeof(GatewayContractInfo).Assembly.GetName().Version?.ToString() ?? GatewayVersionFallback,
+
+                // Convey the negotiated worker-frame maximum so the worker adopts it instead of a
+                // hard-coded default (IPC-02). Sits above the public gRPC cap by the envelope reserve.
+                MaxFrameBytes = (uint)_connection.FrameOptions.MaxMessageBytes,
            });
    }

@@ -985,7 +1061,7 @@ public sealed class WorkerClient : IWorkerClient
    /// <param name="cancellationToken">Cancellation token.</param>
    private async Task WaitForBackgroundTasksAsync(CancellationToken cancellationToken)
    {
-        Task[] tasks = new[] { _readLoopTask, _writeLoopTask, _heartbeatLoopTask }
+        Task[] tasks = new[] { _readLoopTask, _writeLoopTask, _eventWriteLoopTask, _heartbeatLoopTask }
            .Where(task => task is not null)
            .Cast<Task>()
            .ToArray();
@@ -12,4 +12,9 @@ public enum WorkerClientErrorCode
    GatewayShutdown,
    WriteFailed,
    PendingCommandLimitExceeded,
+
+    // The serialized command envelope exceeds the negotiated worker-frame maximum. Rejected at the
+    // enqueue boundary so only the offending command fails (mapped to ResourceExhausted) instead of
+    // the oversized frame reaching the write loop and faulting the whole session (IPC-03).
+    CommandTooLarge,
 }
@@ -10,6 +10,15 @@ public sealed class WorkerFrameProtocolOptions
    /// <summary>Default maximum message size in bytes (16 MB).</summary>
    public const int DefaultMaxMessageBytes = 16 * 1024 * 1024;

+    /// <summary>
+    /// Byte margin the worker-frame (pipe) maximum must keep above the public gRPC message cap so a
+    /// gRPC payload accepted at the public boundary always fits inside one worker frame once wrapped
+    /// in a <c>WorkerEnvelope</c> (correlation id, timestamps, oneof framing). Without this headroom
+    /// the pipe max equals the gRPC max and a maximally-sized accepted request faults the whole
+    /// session on the outbound write (IPC-03). 64 KiB is far larger than the fixed envelope overhead.
+    /// </summary>
+    public const int EnvelopeOverheadReserveBytes = 64 * 1024;
+
    /// <summary>
    /// Initializes worker frame protocol options with a session ID.
    /// </summary>
@@ -38,7 +38,7 @@
      "ShutdownTimeoutSeconds": 10,
      "HeartbeatIntervalSeconds": 5,
      "HeartbeatGraceSeconds": 15,
-      "MaxMessageBytes": 16777216
+      "MaxMessageBytes": 16842752
    },
    "Sessions": {
      "DefaultCommandTimeoutSeconds": 30,
@@ -35,7 +35,8 @@ public sealed class GatewayOptionsTests
        Assert.Equal(10, options.Worker.ShutdownTimeoutSeconds);
        Assert.Equal(5, options.Worker.HeartbeatIntervalSeconds);
        Assert.Equal(15, options.Worker.HeartbeatGraceSeconds);
-        Assert.Equal(16 * 1024 * 1024, options.Worker.MaxMessageBytes);
+        // 16 MiB public gRPC cap plus the 64 KiB worker-frame envelope reserve (IPC-03 headroom).
+        Assert.Equal((16 * 1024 * 1024) + (64 * 1024), options.Worker.MaxMessageBytes);

        Assert.Equal(30, options.Sessions.DefaultCommandTimeoutSeconds);
        Assert.Equal(64, options.Sessions.MaxSessions);
@@ -771,4 +771,50 @@ public sealed class GatewayOptionsValidatorTests
        Assert.True(result.Failed);
        Assert.Contains(result.Failures!, f => f.Contains("ApiKeyFailureTrackedPeers"));
    }
+
+    private static GatewayOptions WithWorkerAndProtocol(WorkerOptions worker, ProtocolOptions protocol)
+    {
+        GatewayOptions source = ValidOptions();
+        return new GatewayOptions
+        {
+            Authentication = source.Authentication,
+            Ldap = source.Ldap,
+            Worker = worker,
+            Sessions = source.Sessions,
+            Events = source.Events,
+            Dashboard = source.Dashboard,
+            Protocol = protocol,
+            Alarms = source.Alarms,
+            Tls = source.Tls,
+        };
+    }
+
+    /// <summary>
+    /// Verifies the default worker-frame maximum keeps the required envelope-overhead reserve above
+    /// the default public gRPC cap, so a stock configuration passes the IPC-03 headroom check.
+    /// </summary>
+    [Fact]
+    public void Validate_Succeeds_WhenWorkerFrameMaxHasEnvelopeHeadroom()
+    {
+        ValidateOptionsResult result = new GatewayOptionsValidator().Validate(null, ValidOptions());
+        Assert.True(result.Succeeded);
+    }
+
+    /// <summary>
+    /// Verifies that a worker-frame maximum equal to the gRPC cap (zero headroom) fails validation:
+    /// a maximally-sized accepted gRPC payload would not fit one worker frame once wrapped in a
+    /// WorkerEnvelope, faulting the whole session on the outbound write (IPC-03).
+    /// </summary>
+    [Fact]
+    public void Validate_Fails_WhenWorkerFrameMaxEqualsGrpcMaxWithoutHeadroom()
+    {
+        const int grpcMax = 16 * 1024 * 1024;
+        ValidateOptionsResult result = new GatewayOptionsValidator().Validate(
+            null,
+            WithWorkerAndProtocol(
+                new WorkerOptions { MaxMessageBytes = grpcMax },
+                new ProtocolOptions { MaxGrpcMessageBytes = grpcMax }));
+        Assert.True(result.Failed);
+        Assert.Contains(result.Failures!, f => f.Contains("MaxMessageBytes") && f.Contains("reserve"));
+    }
 }
@@ -1,6 +1,7 @@
 using Microsoft.Extensions.Diagnostics.HealthChecks;
 using ZB.MOM.WW.Auth.ApiKeys.Sqlite;
 using ZB.MOM.WW.MxGateway.Server.Diagnostics;
+using ZB.MOM.WW.MxGateway.Tests.Security.Authentication;

 namespace ZB.MOM.WW.MxGateway.Tests.Diagnostics;

@@ -17,14 +18,14 @@ public sealed class AuthStoreHealthCheckTests
    [Fact]
    public async Task Healthy_WhenStoreReachable()
    {
-        var path = Path.Combine(Path.GetTempPath(), $"authcheck-{Guid.NewGuid():N}.db");
-        try
-        {
-            var check = new AuthStoreHealthCheck(FactoryFor(path));
-            var result = await check.CheckHealthAsync(new HealthCheckContext());
-            Assert.Equal(HealthStatus.Healthy, result.Status);
-        }
-        finally { if (File.Exists(path)) File.Delete(path); }
+        // Open a real SQLite file via the health check. TempDatabaseDirectory clears the
+        // Microsoft.Data.Sqlite connection pool on dispose before deleting the file; without
+        // that the pool keeps the .db handle open and the delete throws "used by another
+        // process" on Windows (a latent full-suite flake — the file opens fine on macOS).
+        using TempDatabaseDirectory directory = TempDatabaseDirectory.Create("authcheck");
+        var check = new AuthStoreHealthCheck(FactoryFor(directory.DatabasePath()));
+        var result = await check.CheckHealthAsync(new HealthCheckContext());
+        Assert.Equal(HealthStatus.Healthy, result.Status);
    }

    /// <summary>The health check reports unhealthy when the database path cannot be opened.</summary>
@@ -128,11 +128,58 @@ public sealed class DashboardAuthorizationHandlerTests
        Assert.False(context.HasSucceeded);
    }

+    /// <summary>
+    /// Verifies that the anonymous-localhost bypass is read-only: a loopback request with
+    /// <c>AllowAnonymousLocalhost</c> on is denied the <see cref="DashboardAuthorizationRequirement.AdminOnly"/>
+    /// requirement, so destructive/admin surfaces are never reachable without a real Admin role.
+    /// </summary>
+    /// <returns>A task that represents the asynchronous operation.</returns>
+    [Fact]
+    public async Task HandleAsync_AnonymousLocalhostAllowed_DoesNotSatisfyAdminPolicy()
+    {
+        AuthorizationHandlerContext context = await AuthorizeAsync(
+            new ClaimsPrincipal(new ClaimsIdentity()),
+            IPAddress.Loopback,
+            allowAnonymousLocalhost: true,
+            DashboardAuthorizationRequirement.AdminOnly);
+
+        Assert.False(context.HasSucceeded);
+    }
+
+    /// <summary>
+    /// Verifies that with authentication fully disabled the environment bypass stays read-only:
+    /// a remote request satisfies <see cref="DashboardAuthorizationRequirement.AnyDashboardRole"/>
+    /// (Viewer) but is denied <see cref="DashboardAuthorizationRequirement.AdminOnly"/>.
+    /// </summary>
+    /// <returns>A task that represents the asynchronous operation.</returns>
+    [Fact]
+    public async Task HandleAsync_AuthenticationDisabled_GrantsViewerButNotAdmin()
+    {
+        AuthorizationHandlerContext viewerContext = await AuthorizeAsync(
+            new ClaimsPrincipal(new ClaimsIdentity()),
+            IPAddress.Parse("10.0.0.5"),
+            allowAnonymousLocalhost: false,
+            DashboardAuthorizationRequirement.AnyDashboardRole,
+            authenticationMode: AuthenticationMode.Disabled);
+
+        Assert.True(viewerContext.HasSucceeded);
+
+        AuthorizationHandlerContext adminContext = await AuthorizeAsync(
+            new ClaimsPrincipal(new ClaimsIdentity()),
+            IPAddress.Parse("10.0.0.5"),
+            allowAnonymousLocalhost: false,
+            DashboardAuthorizationRequirement.AdminOnly,
+            authenticationMode: AuthenticationMode.Disabled);
+
+        Assert.False(adminContext.HasSucceeded);
+    }
+
    private static async Task<AuthorizationHandlerContext> AuthorizeAsync(
        ClaimsPrincipal principal,
        IPAddress remoteAddress,
        bool allowAnonymousLocalhost,
-        DashboardAuthorizationRequirement requirement)
+        DashboardAuthorizationRequirement requirement,
+        AuthenticationMode authenticationMode = AuthenticationMode.ApiKey)
    {
        DefaultHttpContext httpContext = new();
        httpContext.Connection.RemoteIpAddress = remoteAddress;
@@ -140,6 +187,10 @@ public sealed class DashboardAuthorizationHandlerTests
            new HttpContextAccessor { HttpContext = httpContext },
            Options.Create(new GatewayOptions
            {
+                Authentication = new AuthenticationOptions
+                {
+                    Mode = authenticationMode,
+                },
                Dashboard = new DashboardOptions
                {
                    AllowAnonymousLocalhost = allowAnonymousLocalhost,
@@ -1,6 +1,8 @@
+using System.Collections.Concurrent;
 using System.Diagnostics.CodeAnalysis;
 using System.Security.Claims;
 using Microsoft.AspNetCore.Http;
+using ZB.MOM.WW.Audit;
 using ZB.MOM.WW.MxGateway.Contracts.Proto;
 using ZB.MOM.WW.MxGateway.Server.Dashboard;
 using ZB.MOM.WW.MxGateway.Server.Sessions;
@@ -198,14 +200,91 @@ public sealed class DashboardSessionAdminServiceTests
        Assert.False(string.IsNullOrWhiteSpace(result.Message));
    }

-    private static DashboardSessionAdminService CreateService(ISessionManager sessionManager)
+    /// <summary>
+    /// Verifies that a successful close writes a canonical <c>dashboard-close-session</c>
+    /// <see cref="AuditEvent"/> — actor, session-id target, and Success outcome — to the
+    /// audit store, not only the operational log (SEC-12).
+    /// </summary>
+    /// <returns>A task that represents the asynchronous operation.</returns>
+    [Fact]
+    public async Task CloseSessionAsync_AdminClose_WritesCanonicalAuditEvent()
+    {
+        FakeSessionManager sessionManager = new();
+        RecordingAuditWriter auditWriter = new();
+        DashboardSessionAdminService service = CreateService(sessionManager, auditWriter);
+
+        DashboardSessionAdminResult result = await service.CloseSessionAsync(
+            CreateUser(DashboardRoles.Admin),
+            "session-1",
+            CancellationToken.None);
+
+        Assert.True(result.Succeeded);
+        AuditEvent audit = Assert.Single(auditWriter.Events);
+        Assert.Equal(DashboardSessionAdminService.CloseSessionAction, audit.Action);
+        Assert.Equal(DashboardSessionAdminService.SessionAdminCategory, audit.Category);
+        Assert.Equal("session-1", audit.Target);
+        Assert.Equal("tester", audit.Actor);
+        Assert.Equal(AuditOutcome.Success, audit.Outcome);
+    }
+
+    /// <summary>
+    /// Verifies that a successful kill writes a canonical <c>dashboard-kill-worker</c>
+    /// <see cref="AuditEvent"/> to the audit store (SEC-12).
+    /// </summary>
+    /// <returns>A task that represents the asynchronous operation.</returns>
+    [Fact]
+    public async Task KillWorkerAsync_AdminKill_WritesCanonicalAuditEvent()
+    {
+        FakeSessionManager sessionManager = new();
+        RecordingAuditWriter auditWriter = new();
+        DashboardSessionAdminService service = CreateService(sessionManager, auditWriter);
+
+        DashboardSessionAdminResult result = await service.KillWorkerAsync(
+            CreateUser(DashboardRoles.Admin),
+            "session-1",
+            CancellationToken.None);
+
+        Assert.True(result.Succeeded);
+        AuditEvent audit = Assert.Single(auditWriter.Events);
+        Assert.Equal(DashboardSessionAdminService.KillWorkerAction, audit.Action);
+        Assert.Equal("session-1", audit.Target);
+        Assert.Equal(AuditOutcome.Success, audit.Outcome);
+    }
+
+    /// <summary>
+    /// Verifies that an unauthorized (viewer) close attempt still writes a <c>Denied</c>
+    /// audit row, so rejected destructive attempts are durably recorded (SEC-12).
+    /// </summary>
+    /// <returns>A task that represents the asynchronous operation.</returns>
+    [Fact]
+    public async Task CloseSessionAsync_ViewerDenied_WritesDeniedAuditEvent()
+    {
+        FakeSessionManager sessionManager = new();
+        RecordingAuditWriter auditWriter = new();
+        DashboardSessionAdminService service = CreateService(sessionManager, auditWriter);
+
+        DashboardSessionAdminResult result = await service.CloseSessionAsync(
+            CreateUser(DashboardRoles.Viewer),
+            "session-1",
+            CancellationToken.None);
+
+        Assert.False(result.Succeeded);
+        AuditEvent audit = Assert.Single(auditWriter.Events);
+        Assert.Equal(DashboardSessionAdminService.CloseSessionAction, audit.Action);
+        Assert.Equal(AuditOutcome.Denied, audit.Outcome);
+    }
+
+    private static DashboardSessionAdminService CreateService(
+        ISessionManager sessionManager,
+        IAuditWriter? auditWriter = null)
    {
        DefaultHttpContext httpContext = new();
        httpContext.Connection.RemoteIpAddress = System.Net.IPAddress.Loopback;

        return new DashboardSessionAdminService(
            sessionManager,
-            new HttpContextAccessor { HttpContext = httpContext });
+            new HttpContextAccessor { HttpContext = httpContext },
+            auditWriter ?? new RecordingAuditWriter());
    }

    private static ClaimsPrincipal CreateUser(string role)
@@ -219,6 +298,21 @@ public sealed class DashboardSessionAdminServiceTests
        return new ClaimsPrincipal(identity);
    }

+    private sealed class RecordingAuditWriter : IAuditWriter
+    {
+        private readonly ConcurrentQueue<AuditEvent> _events = new();
+
+        /// <summary>Gets the audit events written through this writer, in order.</summary>
+        public IReadOnlyList<AuditEvent> Events => _events.ToArray();
+
+        /// <inheritdoc />
+        public Task WriteAsync(AuditEvent evt, CancellationToken ct = default)
+        {
+            _events.Enqueue(evt);
+            return Task.CompletedTask;
+        }
+    }
+
    private sealed class FakeSessionManager : ISessionManager
    {
        /// <summary>Gets the number of times CloseSessionAsync was invoked.</summary>
@@ -9,6 +9,10 @@ using ZB.MOM.WW.MxGateway.Server;

 namespace ZB.MOM.WW.MxGateway.Tests.Gateway;

+// Sets process-global Kestrel/TLS environment variables that GatewayApplication.Build reads;
+// serialized against all other collections so a parallel host-building test cannot inherit them
+// mid-run and race on the generated-certificate file. See GlobalEnvironmentCollection.
+[Collection(TestSupport.GlobalEnvironmentCollection.Name)]
 public sealed class GatewayTlsBootstrapTests
 {
    /// <summary>
@@ -0,0 +1,45 @@
+using Grpc.Core;
+using ZB.MOM.WW.MxGateway.Contracts.Proto;
+using ZB.MOM.WW.MxGateway.Server.Grpc;
+
+namespace ZB.MOM.WW.MxGateway.Tests.Gateway.Grpc;
+
+public sealed class MxAccessGrpcRequestValidatorTests
+{
+    private static MxCommandRequest DrainRequest(uint maxEvents) => new()
+    {
+        SessionId = "session-1",
+        Command = new MxCommand
+        {
+            Kind = MxCommandKind.DrainEvents,
+            DrainEvents = new DrainEventsCommand { MaxEvents = maxEvents },
+        },
+    };
+
+    /// <summary>
+    /// Verifies a DrainEvents request within the per-request ceiling passes validation, including the
+    /// <c>max_events = 0</c> "worker default cap" sentinel (IPC-04).
+    /// </summary>
+    [Theory]
+    [InlineData(0u)]
+    [InlineData(1u)]
+    [InlineData(10_000u)]
+    public void ValidateInvoke_AllowsDrainEvents_WithinCeiling(uint maxEvents)
+    {
+        MxAccessGrpcRequestValidator validator = new();
+        validator.ValidateInvoke(DrainRequest(maxEvents));
+    }
+
+    /// <summary>
+    /// Verifies a DrainEvents request above the per-request ceiling is rejected with InvalidArgument
+    /// so one accepted request cannot pack an unbounded reply frame (IPC-04).
+    /// </summary>
+    [Fact]
+    public void ValidateInvoke_RejectsDrainEvents_AboveCeiling()
+    {
+        MxAccessGrpcRequestValidator validator = new();
+        RpcException exception = Assert.Throws<RpcException>(() => validator.ValidateInvoke(DrainRequest(10_001)));
+        Assert.Equal(StatusCode.InvalidArgument, exception.StatusCode);
+        Assert.Contains("max_events", exception.Status.Detail, StringComparison.Ordinal);
+    }
+}
@@ -56,6 +56,52 @@ public sealed class WorkerClientTests
        Assert.Equal(MxCommandKind.Ping, reply.Reply.Kind);
    }

+    /// <summary>
+    /// Verifies that a command whose serialized envelope exceeds the negotiated worker-frame maximum
+    /// fails only that command with <see cref="WorkerClientErrorCode.CommandTooLarge"/> at the enqueue
+    /// boundary, leaving the client ready for subsequent commands (IPC-03). Without the pre-check the
+    /// oversized frame would reach the write loop and fault the whole session.
+    /// </summary>
+    /// <returns>A task that represents the asynchronous operation.</returns>
+    [Fact]
+    public async Task InvokeAsync_WhenCommandExceedsFrameMax_FailsOnlyThatCommandAndStaysReady()
+    {
+        await using PipePair pipePair = await PipePair.CreateAsync();
+        await using WorkerClient client = CreateClient(pipePair, maxMessageBytes: 4096);
+        await CompleteHandshakeAsync(client, pipePair);
+
+        WorkerCommand oversized = new()
+        {
+            Command = new MxCommand
+            {
+                Kind = MxCommandKind.Write,
+                Write = new WriteCommand
+                {
+                    ServerHandle = 1,
+                    ItemHandle = 2,
+                    Value = new MxValue { StringValue = new string('x', 8192) },
+                },
+            },
+        };
+
+        WorkerClientException exception = await Assert.ThrowsAsync<WorkerClientException>(
+            async () => await client.InvokeAsync(oversized, TestTimeout, CancellationToken.None));
+        Assert.Equal(WorkerClientErrorCode.CommandTooLarge, exception.ErrorCode);
+        Assert.Equal(WorkerClientState.Ready, client.State);
+
+        // A subsequent normally-sized command still round-trips: the session was not faulted.
+        Task<WorkerCommandReply> nextInvoke = client.InvokeAsync(
+            CreateCommand(MxCommandKind.Ping),
+            TestTimeout,
+            CancellationToken.None);
+        WorkerEnvelope commandEnvelope = await pipePair.WorkerReader.ReadAsync().AsTask().WaitAsync(TestTimeout);
+        await pipePair.WorkerWriter.WriteAsync(
+            CreateCommandReplyEnvelope(commandEnvelope.CorrelationId, MxCommandKind.Ping));
+        WorkerCommandReply reply = await nextInvoke.WaitAsync(TestTimeout);
+        Assert.Equal(MxCommandKind.Ping, reply.Reply.Kind);
+        Assert.Equal(WorkerClientState.Ready, client.State);
+    }
+
    /// <summary>Verifies that InvokeAsync ignores late replies and keeps the client ready.</summary>
    /// <returns>A task that represents the asynchronous operation.</returns>
    [Fact]
@@ -172,6 +218,51 @@ public sealed class WorkerClientTests
        Assert.Equal(WorkerClientState.Faulted, client.State);
    }

+    /// <summary>
+    /// Verifies that a command reply arriving on the pipe after events is dispatched promptly even
+    /// when the event channel is full and has no consumer — event enqueue is decoupled from the read
+    /// loop, so a blocked event writer cannot delay a reply (GWC-04). The event full-mode timeout is
+    /// set far above the command timeout: without the decoupling the read loop would block behind the
+    /// full event channel and the in-flight InvokeAsync would hit CommandTimeout.
+    /// </summary>
+    /// <returns>A task that represents the asynchronous operation.</returns>
+    [Fact]
+    public async Task ReadLoop_WhenEventChannelFull_DispatchesCommandReplyWithoutBlocking()
+    {
+        await using PipePair pipePair = await PipePair.CreateAsync();
+        await using WorkerClient client = CreateClient(
+            pipePair,
+            new WorkerClientOptions
+            {
+                EventChannelCapacity = 1,
+                // Much larger than TestTimeout: the read loop must not block for this long behind the
+                // full event channel while a reply waits.
+                EventChannelFullModeTimeout = TimeSpan.FromSeconds(30),
+                HeartbeatGrace = TimeSpan.FromSeconds(60),
+                HeartbeatCheckInterval = TimeSpan.FromSeconds(60),
+            });
+        await CompleteHandshakeAsync(client, pipePair);
+
+        // No StreamEvents consumer is attached, so the capacity-1 event channel fills and the event
+        // writer blocks on the second event's timed WriteAsync.
+        await pipePair.WorkerWriter.WriteAsync(
+            CreateEventEnvelope(sequence: 11, MxEventFamily.OnDataChange));
+        await pipePair.WorkerWriter.WriteAsync(
+            CreateEventEnvelope(sequence: 12, MxEventFamily.OnDataChange));
+
+        Task<WorkerCommandReply> invokeTask = client.InvokeAsync(
+            CreateCommand(MxCommandKind.Ping),
+            TestTimeout,
+            CancellationToken.None);
+        WorkerEnvelope commandEnvelope = await pipePair.WorkerReader.ReadAsync().AsTask().WaitAsync(TestTimeout);
+        await pipePair.WorkerWriter.WriteAsync(
+            CreateCommandReplyEnvelope(commandEnvelope.CorrelationId, MxCommandKind.Ping));
+
+        WorkerCommandReply reply = await invokeTask.WaitAsync(TestTimeout);
+        Assert.Equal(MxCommandKind.Ping, reply.Reply.Kind);
+        Assert.Equal(WorkerClientState.Ready, client.State);
+    }
+
    /// <summary>
    /// Verifies that when the client faults it kills the owned worker process.
    /// The assertion waits on <see cref="FakeWorkerProcess.WaitForExitAsync"/>, which
@@ -564,9 +655,13 @@ public sealed class WorkerClientTests
        WorkerClientOptions? options = null,
        GatewayMetrics? metrics = null,
        WorkerProcessHandle? processHandle = null,
-        TimeProvider? timeProvider = null)
+        TimeProvider? timeProvider = null,
+        int maxMessageBytes = WorkerFrameProtocolOptions.DefaultMaxMessageBytes)
    {
-        WorkerFrameProtocolOptions frameOptions = new(SessionId);
+        WorkerFrameProtocolOptions frameOptions = new(
+            SessionId,
+            GatewayContractInfo.WorkerProtocolVersion,
+            maxMessageBytes);
        WorkerClientConnection connection = new(
            SessionId,
            Nonce,
@@ -157,6 +157,56 @@ public sealed class GatewayMetricsTests
        Assert.Equal(2, capturedMode);
    }

+    /// <summary>
+    /// Verifies that <see cref="GatewayMetrics.HeartbeatFailed"/> increments
+    /// <c>mxgateway.heartbeats.failed</c> without emitting a <c>session_id</c> tag: the tag is
+    /// unbounded cardinality (every session mints a new exporter time series), so per-session
+    /// attribution is deliberately kept out of the exported counter (SEC-20).
+    /// </summary>
+    [Fact]
+    public void HeartbeatFailed_IncrementsCounterWithoutSessionIdTag()
+    {
+        using GatewayMetrics metrics = new();
+        using MeterListener listener = new();
+
+        long capturedValue = 0;
+        bool sawSessionIdTag = false;
+
+        listener.InstrumentPublished = (instrument, meterListener) =>
+        {
+            if (ReferenceEquals(instrument.Meter, metrics.Meter)
+                && instrument.Name == "mxgateway.heartbeats.failed")
+            {
+                meterListener.EnableMeasurementEvents(instrument);
+            }
+        };
+        listener.SetMeasurementEventCallback<long>(
+            (instrument, measurement, tags, _) =>
+            {
+                if (!ReferenceEquals(instrument.Meter, metrics.Meter)
+                    || instrument.Name != "mxgateway.heartbeats.failed")
+                {
+                    return;
+                }
+
+                capturedValue += measurement;
+                foreach (KeyValuePair<string, object?> tag in tags)
+                {
+                    if (tag.Key == "session_id")
+                    {
+                        sawSessionIdTag = true;
+                    }
+                }
+            });
+        listener.Start();
+
+        metrics.HeartbeatFailed("session-1");
+
+        Assert.Equal(1, capturedValue);
+        Assert.False(sawSessionIdTag);
+        Assert.Equal(1, metrics.GetSnapshot().HeartbeatFailures);
+    }
+
    /// <summary>Verifies that removing session events only affects that session.</summary>
    [Fact]
    public void RemoveSessionEvents_RemovesOnlyThatSession()
@@ -0,0 +1,18 @@
+namespace ZB.MOM.WW.MxGateway.Tests.TestSupport;
+
+/// <summary>
+/// xUnit collection for tests that mutate <em>process-global</em> state (environment variables
+/// read by <c>GatewayApplication.Build</c>, e.g. <c>Kestrel__Endpoints__…</c> and
+/// <c>MxGateway__Tls__SelfSignedCertPath</c>). <c>DisableParallelization</c> keeps such a test from
+/// running concurrently with any other collection: otherwise a parallel host-building test inherits
+/// the mutated variables mid-run and the two race on the same generated-certificate file (on Windows,
+/// "the process cannot access the file … because it is being used by another process"). Membership is
+/// deliberately narrow — only add classes that set/clear real environment variables, not ones that
+/// pass configuration through <c>Build([...])</c> command-line args.
+/// </summary>
+[CollectionDefinition(Name, DisableParallelization = true)]
+public sealed class GlobalEnvironmentCollection
+{
+    /// <summary>The collection name applied via <c>[Collection(GlobalEnvironmentCollection.Name)]</c>.</summary>
+    public const string Name = "GlobalEnvironmentMutation";
+}
@@ -3,7 +3,8 @@ using System.Runtime.CompilerServices;
 namespace ZB.MOM.WW.MxGateway.Tests.TestSupport;

 /// <summary>
-/// Defaults the host environment to Development for the whole test assembly.
+/// Defaults the host environment to Development and isolates the test process's on-disk
+/// gateway paths, for the whole test assembly.
 /// </summary>
 /// <remarks>
 /// Many tests build the full gateway host through <c>GatewayApplication.Build</c> against the dev
@@ -15,6 +16,15 @@ namespace ZB.MOM.WW.MxGateway.Tests.TestSupport;
 /// <c>GatewayOptionsValidator</c> with its <c>isProduction</c> constructor (no host environment) and
 /// are unaffected; a test that needs Production can still pass <c>--environment=Production</c>, which
 /// overrides this default.
+/// <para>
+/// The self-signed-cert path is also defaulted to a per-process temp file. Otherwise every
+/// full-host test that triggers HTTPS-cert generation writes the shared
+/// <c>CommonApplicationData/MxGateway/certs/gateway-selfsigned.pfx</c> default — parallel xUnit
+/// test classes then collide on that path's temp file (Windows: "the process cannot access the
+/// file ... because it is being used by another process"), and on a shared CI/dev box the suite
+/// also fights the deployed gateway service for the same file. Pointing at a per-process path
+/// isolates the suite: the first host-building test generates the cert, the rest load it.
+/// </para>
 /// </remarks>
 internal static class TestHostEnvironmentInitializer
 {
@@ -25,5 +35,17 @@ internal static class TestHostEnvironmentInitializer
        {
            Environment.SetEnvironmentVariable("ASPNETCORE_ENVIRONMENT", "Development");
        }
+
+        if (string.IsNullOrEmpty(Environment.GetEnvironmentVariable("MxGateway__Tls__SelfSignedCertPath")))
+        {
+            // ProcessId keeps the path stable within this test process (so a valid cert
+            // generated by the first host-building test is reused by the rest) yet unique
+            // across processes (so concurrent test runs / the deployed service never share it).
+            string certPath = Path.Combine(
+                Path.GetTempPath(),
+                $"mxgw-tests-{Environment.ProcessId}",
+                "gateway-selfsigned.pfx");
+            Environment.SetEnvironmentVariable("MxGateway__Tls__SelfSignedCertPath", certPath);
+        }
    }
 }
@@ -1,5 +1,7 @@
+using System;
 using System.IO;
 using System.Linq;
+using System.Threading;
 using System.Threading.Tasks;
 using ZB.MOM.WW.MxGateway.Contracts;
 using ZB.MOM.WW.MxGateway.Contracts.Proto;
@@ -305,6 +307,101 @@ public sealed class WorkerFrameProtocolTests
            Nonce);
    }

+    /// <summary>
+    /// Verifies that under concurrent writers every frame receives a distinct, gap-free sequence in
+    /// strictly increasing on-wire order — the sequence is stamped by the writer at write time, so the
+    /// wire order and the stamped sequence always agree (WRK-04).
+    /// </summary>
+    /// <returns>A task that represents the asynchronous operation.</returns>
+    [Fact]
+    public async Task WriteAsync_UnderConcurrentCalls_StampsGapFreeMonotonicSequence()
+    {
+        const int frameCount = 50;
+        WorkerFrameProtocolOptions options = CreateOptions();
+        using MemoryStream stream = new();
+        WorkerFrameWriter writer = new(stream, options);
+
+        await Task.WhenAll(
+            Enumerable.Range(0, frameCount).Select(_ => writer.WriteAsync(CreateEventEnvelope())));
+
+        stream.Position = 0;
+        WorkerFrameReader reader = new(stream, options);
+        ulong[] sequences = new ulong[frameCount];
+        for (int index = 0; index < frameCount; index++)
+        {
+            sequences[index] = (await reader.ReadAsync()).Sequence;
+        }
+
+        // On-wire order is strictly increasing 1..frameCount with no gaps or duplicates.
+        Assert.Equal(Enumerable.Range(1, frameCount).Select(value => (ulong)value), sequences);
+    }
+
+    /// <summary>
+    /// Verifies that when a control frame and an event frame are both queued behind an in-progress
+    /// write, the draining lock-holder writes the control frame first even though the event was queued
+    /// earlier (WRK-07).
+    /// </summary>
+    /// <returns>A task that represents the asynchronous operation.</returns>
+    [Fact]
+    public async Task WriteAsync_WhenControlAndEventQueuedTogether_WritesControlFirst()
+    {
+        WorkerFrameProtocolOptions options = CreateOptions();
+        using GatedWriteStream stream = new();
+        WorkerFrameWriter writer = new(stream, options);
+
+        // First write occupies the writer and blocks inside the stream, holding the write lock.
+        Task firstWrite = writer.WriteAsync(CreateGatewayHelloEnvelope(), WorkerFrameWritePriority.Control);
+        await AwaitWithTimeoutAsync(stream.FirstWriteStarted);
+
+        // Queue an event first, then a control frame, while the writer is blocked. Both wait for the lock.
+        Task eventWrite = writer.WriteAsync(CreateEventEnvelope(), WorkerFrameWritePriority.Event);
+        Task controlWrite = writer.WriteAsync(CreateGatewayHelloEnvelope(), WorkerFrameWritePriority.Control);
+        await Task.Delay(50);
+
+        stream.ReleaseFirstWrite();
+        await AwaitWithTimeoutAsync(Task.WhenAll(firstWrite, eventWrite, controlWrite));
+
+        stream.Position = 0;
+        WorkerFrameReader reader = new(stream, options);
+        WorkerEnvelope frame1 = await reader.ReadAsync();
+        WorkerEnvelope frame2 = await reader.ReadAsync();
+        WorkerEnvelope frame3 = await reader.ReadAsync();
+
+        Assert.Equal(WorkerEnvelope.BodyOneofCase.GatewayHello, frame1.BodyCase);
+        // The control frame jumped ahead of the earlier-queued event.
+        Assert.Equal(WorkerEnvelope.BodyOneofCase.GatewayHello, frame2.BodyCase);
+        Assert.Equal(WorkerEnvelope.BodyOneofCase.WorkerEvent, frame3.BodyCase);
+    }
+
+    /// <summary>Verifies a zero negotiated frame maximum keeps the constructor default (IPC-02).</summary>
+    [Fact]
+    public void AdoptNegotiatedMaxMessageBytes_WithZero_KeepsDefault()
+    {
+        WorkerFrameProtocolOptions options = CreateOptions();
+        int original = options.MaxMessageBytes;
+        options.AdoptNegotiatedMaxMessageBytes(0);
+        Assert.Equal(original, options.MaxMessageBytes);
+    }
+
+    /// <summary>Verifies an in-range negotiated frame maximum is adopted (IPC-02).</summary>
+    [Fact]
+    public void AdoptNegotiatedMaxMessageBytes_WithInRangeValue_Adopts()
+    {
+        WorkerFrameProtocolOptions options = CreateOptions();
+        options.AdoptNegotiatedMaxMessageBytes(4 * 1024 * 1024);
+        Assert.Equal(4 * 1024 * 1024, options.MaxMessageBytes);
+    }
+
+    /// <summary>Verifies a negotiated frame maximum above the worker ceiling is rejected (IPC-02).</summary>
+    [Fact]
+    public void AdoptNegotiatedMaxMessageBytes_AboveCeiling_Throws()
+    {
+        WorkerFrameProtocolOptions options = CreateOptions();
+        WorkerFrameProtocolException exception = Assert.Throws<WorkerFrameProtocolException>(
+            () => options.AdoptNegotiatedMaxMessageBytes((uint)WorkerFrameProtocolOptions.MaxNegotiableFrameBytes + 1));
+        Assert.Equal(WorkerFrameProtocolErrorCode.InvalidConfiguration, exception.ErrorCode);
+    }
+
    private static WorkerEnvelope CreateGatewayHelloEnvelope(ulong sequence = 1)
    {
        return new WorkerEnvelope
@@ -321,4 +418,63 @@ public sealed class WorkerFrameProtocolTests
        };
    }

+    // net48 has no Task.WaitAsync(TimeSpan); fail the test rather than hang if the writer misbehaves.
+    private static async Task AwaitWithTimeoutAsync(Task task)
+    {
+        Task completed = await Task.WhenAny(task, Task.Delay(TimeSpan.FromSeconds(5)));
+        if (completed != task)
+        {
+            throw new TimeoutException("Timed out waiting for the frame writer.");
+        }
+
+        await task;
+    }
+
+    private static WorkerEnvelope CreateEventEnvelope()
+    {
+        return new WorkerEnvelope
+        {
+            ProtocolVersion = GatewayContractInfo.WorkerProtocolVersion,
+            SessionId = SessionId,
+            WorkerEvent = new WorkerEvent
+            {
+                Event = new MxEvent { SessionId = SessionId },
+            },
+        };
+    }
+
+    // A MemoryStream whose first WriteAsync blocks until released, so a test can queue additional frames
+    // behind an in-progress write and observe the writer's priority ordering.
+    private sealed class GatedWriteStream : MemoryStream
+    {
+        private readonly SemaphoreSlim _release = new SemaphoreSlim(0);
+        private readonly TaskCompletionSource<bool> _firstWriteStarted =
+            new TaskCompletionSource<bool>(TaskCreationOptions.RunContinuationsAsynchronously);
+        private int _writeCount;
+
+        public Task FirstWriteStarted => _firstWriteStarted.Task;
+
+        public void ReleaseFirstWrite() => _release.Release();
+
+        public override async Task WriteAsync(byte[] buffer, int offset, int count, CancellationToken cancellationToken)
+        {
+            if (Interlocked.Increment(ref _writeCount) == 1)
+            {
+                _firstWriteStarted.TrySetResult(true);
+                await _release.WaitAsync(cancellationToken);
+            }
+
+            await base.WriteAsync(buffer, offset, count, cancellationToken);
+        }
+
+        protected override void Dispose(bool disposing)
+        {
+            if (disposing)
+            {
+                _release.Dispose();
+            }
+
+            base.Dispose(disposing);
+        }
+    }
 }
@@ -449,6 +449,44 @@ public sealed class WorkerPipeSessionTests
        await SendShutdownAndWaitAsync(pipePair, runTask, cancellation.Token);
    }

+    /// <summary>
+    ///     Verifies that a DrainEvents control command with <c>max_events = 0</c> is bounded by the
+    ///     worker rather than draining the entire queue into one reply frame: the session passes a
+    ///     capped, non-zero maximum to the runtime session (IPC-04).
+    /// </summary>
+    /// <returns>A task that represents the asynchronous operation.</returns>
+    [Fact]
+    public async Task RunAsync_DrainEventsWithZeroMaxEvents_BoundsTheDrain()
+    {
+        using CancellationTokenSource cancellation = new(TimeSpan.FromSeconds(5));
+        using PipePair pipePair = await PipePair.CreateAsync(cancellation.Token);
+        FakeRuntimeSession runtime = new() { SuppressDrainForBatchSize = 128 };
+        WorkerPipeSession session = CreatePipeSession(pipePair.WorkerStream, runtime);
+        runtime.EnqueueEvent(CreateWorkerEvent(sequence: 11));
+        Task runTask = session.RunAsync(cancellation.Token);
+        await CompleteGatewayHandshakeAsync(pipePair, cancellation.Token);
+
+        await pipePair.GatewayWriter
+            .WriteAsync(
+                CreateControlCommandEnvelope(
+                    "drain-cap-1",
+                    MxCommandKind.DrainEvents,
+                    command => command.DrainEvents = new DrainEventsCommand { MaxEvents = 0 }),
+                cancellation.Token);
+
+        await ReadUntilAsync(
+            pipePair.GatewayReader,
+            WorkerEnvelope.BodyOneofCase.WorkerCommandReply,
+            cancellation.Token);
+
+        // The client asked for "all" (0) but the worker must pass a bounded, non-zero cap so the reply
+        // frame cannot grow without limit.
+        Assert.NotNull(runtime.LastDrainMaxEvents);
+        Assert.NotEqual(0u, runtime.LastDrainMaxEvents!.Value);
+
+        await SendShutdownAndWaitAsync(pipePair, runTask, cancellation.Token);
+    }
+
    /// <summary>
    ///     Verifies that ShutdownWorker returns its OK reply BEFORE the graceful
    ///     shutdown runs and disposes the runtime session, and that the message
@@ -125,6 +125,14 @@ internal sealed class FakeRuntimeSession : IWorkerRuntimeSession
    /// </summary>
    public uint? SuppressDrainForBatchSize { get; set; }

+    /// <summary>
+    ///     Records the <c>maxEvents</c> argument of the most recent non-suppressed
+    ///     <see cref="DrainEvents"/> call — i.e. the effective cap the session passed for an explicit
+    ///     DrainEvents control command. Lets a test assert the worker bounds the drain (IPC-04) rather
+    ///     than forwarding the client's raw <c>max_events = 0</c>.
+    /// </summary>
+    public uint? LastDrainMaxEvents { get; private set; }
+
    /// <inheritdoc />
    public IReadOnlyList<WorkerEvent> DrainEvents(uint maxEvents)
    {
@@ -133,6 +141,8 @@ internal sealed class FakeRuntimeSession : IWorkerRuntimeSession
            return Array.Empty<WorkerEvent>();
        }

+        LastDrainMaxEvents = maxEvents;
+
        lock (gate)
        {
            int drainCount = maxEvents == 0
@@ -10,6 +10,14 @@ public sealed class WorkerFrameProtocolOptions
    /// <summary>Default maximum message size in bytes (16 MB).</summary>
    public const int DefaultMaxMessageBytes = 16 * 1024 * 1024;

+    /// <summary>
+    /// Upper ceiling the worker will accept for a gateway-negotiated frame maximum
+    /// (<c>GatewayHello.max_frame_bytes</c>, IPC-02). Matches the gateway's own configuration ceiling
+    /// so a nonsensical negotiated value is rejected at the handshake rather than driving an absurd
+    /// per-frame allocation. 256 MiB.
+    /// </summary>
+    public const int MaxNegotiableFrameBytes = 256 * 1024 * 1024;
+
    /// <summary>Initializes a new instance of the WorkerFrameProtocolOptions class from WorkerOptions.</summary>
    /// <param name="options">Worker initialization options.</param>
    public WorkerFrameProtocolOptions(WorkerOptions options)
@@ -98,6 +106,36 @@ public sealed class WorkerFrameProtocolOptions
    /// <summary>Gets the nonce for startup validation.</summary>
    public string Nonce { get; }

-    /// <summary>Gets the maximum message size in bytes.</summary>
-    public int MaxMessageBytes { get; }
+    /// <summary>
+    /// Gets the maximum worker-frame message size in bytes. Initialized from the constructor and
+    /// then adopted once from the gateway-negotiated value during the startup handshake
+    /// (<c>GatewayHello.max_frame_bytes</c>, IPC-02) via <see cref="AdoptNegotiatedMaxMessageBytes"/>,
+    /// before the message loop starts. Not mutated afterwards, so the single-threaded handshake write
+    /// is safe for the reader/writer that share this instance.
+    /// </summary>
+    public int MaxMessageBytes { get; private set; }
+
+    /// <summary>
+    /// Adopts the gateway-negotiated frame maximum conveyed in <c>GatewayHello.max_frame_bytes</c>
+    /// (IPC-02). A value of 0 (an older gateway that never set the field) is ignored and the
+    /// constructor default is kept. A value above <see cref="MaxNegotiableFrameBytes"/> is rejected.
+    /// </summary>
+    /// <param name="negotiatedMaxFrameBytes">The gateway-negotiated maximum, or 0 for "keep default".</param>
+    internal void AdoptNegotiatedMaxMessageBytes(uint negotiatedMaxFrameBytes)
+    {
+        if (negotiatedMaxFrameBytes == 0)
+        {
+            return;
+        }
+
+        if (negotiatedMaxFrameBytes > MaxNegotiableFrameBytes)
+        {
+            throw new WorkerFrameProtocolException(
+                WorkerFrameProtocolErrorCode.InvalidConfiguration,
+                $"GatewayHello negotiated frame maximum {negotiatedMaxFrameBytes} exceeds the worker ceiling "
+                + $"of {MaxNegotiableFrameBytes} bytes.");
+        }
+
+        MaxMessageBytes = (int)negotiatedMaxFrameBytes;
+    }
 }
@@ -0,0 +1,17 @@
+namespace ZB.MOM.WW.MxGateway.Worker.Ipc;
+
+/// <summary>
+/// Relative scheduling priority for an outbound worker frame. The single writer task drains all
+/// pending <see cref="Control"/> frames before any <see cref="Event"/> frame, so a command reply,
+/// fault, heartbeat, or shutdown acknowledgement is not delayed behind a backlog of queued events
+/// (WRK-07). Priority only reorders the write; the frame sequence is stamped at actual write time,
+/// so the on-wire order and the envelope <c>Sequence</c> always agree (WRK-04).
+/// </summary>
+public enum WorkerFrameWritePriority
+{
+    /// <summary>Control-plane frame (hello, ready, command reply, heartbeat, fault, shutdown ack). Written ahead of events.</summary>
+    Control = 0,
+
+    /// <summary>Event frame. Written only when no control frame is pending.</summary>
+    Event = 1,
+}
@@ -1,4 +1,5 @@
 using System;
+using System.Collections.Generic;
 using System.IO;
 using System.Threading;
 using System.Threading.Tasks;
@@ -7,12 +8,40 @@ using ZB.MOM.WW.MxGateway.Contracts.Proto;

 namespace ZB.MOM.WW.MxGateway.Worker.Ipc;

-/// <summary>Writes worker frames to a stream with length-prefixed protobuf serialization.</summary>
+/// <summary>
+/// Writes worker frames to a stream with length-prefixed protobuf serialization. Callers enqueue a
+/// frame at a <see cref="WorkerFrameWritePriority"/> and then contend for a single write lock; whoever
+/// holds the lock drains every queued frame, control frames first, so a reply, fault, or heartbeat is
+/// never delayed behind an event backlog (WRK-07). The envelope <c>Sequence</c> is stamped by the
+/// draining lock-holder at the moment of writing, so the on-wire order and the stamped sequence always
+/// agree even under concurrent callers and priority reordering (WRK-04).
+/// </summary>
 public sealed class WorkerFrameWriter
 {
+    private sealed class PendingFrame
+    {
+        public PendingFrame(WorkerEnvelope envelope)
+        {
+            Envelope = envelope;
+            Completion = new TaskCompletionSource<bool>(TaskCreationOptions.RunContinuationsAsynchronously);
+        }
+
+        public WorkerEnvelope Envelope { get; }
+
+        public TaskCompletionSource<bool> Completion { get; }
+    }
+
    private readonly WorkerFrameProtocolOptions _options;
-    private readonly SemaphoreSlim _writeLock = new(1, 1);
+    private readonly SemaphoreSlim _writeLock = new SemaphoreSlim(1, 1);
    private readonly Stream _stream;
+    private readonly object _gate = new object();
+    private readonly Queue<PendingFrame> _controlFrames = new Queue<PendingFrame>();
+    private readonly Queue<PendingFrame> _eventFrames = new Queue<PendingFrame>();
+
+    // Only ever read/written by the current write-lock holder while draining, so no interlock is
+    // needed. Starts at 0 and is pre-incremented, so the first written frame carries sequence 1
+    // (matching the previous behaviour).
+    private ulong _nextSequence;

    /// <summary>Initializes a new instance of the WorkerFrameWriter class.</summary>
    /// <param name="stream">Stream to write frames to.</param>
@@ -25,12 +54,25 @@ public sealed class WorkerFrameWriter
        _options = options ?? throw new ArgumentNullException(nameof(options));
    }

-    /// <summary>Writes a worker envelope frame to the stream with length prefix.</summary>
+    /// <summary>Writes a control-priority worker envelope frame to the stream with length prefix.</summary>
    /// <param name="envelope">Worker envelope to write.</param>
    /// <param name="cancellationToken">Token to cancel the asynchronous operation.</param>
-    /// <returns>A task that represents the asynchronous operation.</returns>
+    /// <returns>A task that completes when the frame has been written and flushed.</returns>
+    public Task WriteAsync(
+        WorkerEnvelope envelope,
+        CancellationToken cancellationToken = default)
+    {
+        return WriteAsync(envelope, WorkerFrameWritePriority.Control, cancellationToken);
+    }
+
+    /// <summary>Queues a worker envelope frame for writing at the given priority and drains the queue.</summary>
+    /// <param name="envelope">Worker envelope to write.</param>
+    /// <param name="priority">Scheduling priority; control frames are written ahead of event frames.</param>
+    /// <param name="cancellationToken">Token to cancel waiting for the write lock.</param>
+    /// <returns>A task that completes when the frame has been written and flushed.</returns>
    public async Task WriteAsync(
        WorkerEnvelope envelope,
+        WorkerFrameWritePriority priority,
        CancellationToken cancellationToken = default)
    {
        if (envelope is null)
@@ -38,8 +80,120 @@ public sealed class WorkerFrameWriter
            throw new ArgumentNullException(nameof(envelope));
        }

+        PendingFrame frame = new PendingFrame(envelope);
+        lock (_gate)
+        {
+            if (priority == WorkerFrameWritePriority.Event)
+            {
+                _eventFrames.Enqueue(frame);
+            }
+            else
+            {
+                _controlFrames.Enqueue(frame);
+            }
+        }
+
+        // Contend for the single writer: whoever wins drains every currently-queued frame in priority
+        // order, so this frame is written by this call or by a concurrent caller that got the lock
+        // first. Either way it completes via its own TaskCompletionSource.
+        await _writeLock.WaitAsync(cancellationToken).ConfigureAwait(false);
+        try
+        {
+            await DrainQueuedFramesAsync().ConfigureAwait(false);
+        }
+        finally
+        {
+            _writeLock.Release();
+        }
+
+        await frame.Completion.Task.ConfigureAwait(false);
+    }
+
+    // Runs only under _writeLock. Drains control frames before event frames, stamping and writing each.
+    // The stream write itself is not cancellable: a frame is written atomically or fails, never left
+    // half-written on the pipe because a caller gave up waiting.
+    private async Task DrainQueuedFramesAsync()
+    {
+        while (true)
+        {
+            PendingFrame? frame = DequeueNext();
+            if (frame is null)
+            {
+                return;
+            }
+
+            try
+            {
+                await WriteFrameAsync(frame.Envelope).ConfigureAwait(false);
+                frame.Completion.TrySetResult(true);
+            }
+            catch (WorkerFrameProtocolException exception) when (IsPerFrameRejection(exception))
+            {
+                // Validation, empty-payload, and oversized-frame errors are specific to this frame and
+                // do not damage the stream; fail only this frame and keep draining the rest.
+                frame.Completion.TrySetException(exception);
+            }
+            catch (Exception exception)
+            {
+                // A stream write/flush failure means the pipe is broken; fail this frame and every frame
+                // still queued so no caller awaits forever, then stop draining.
+                frame.Completion.TrySetException(exception);
+                FailAllQueued(exception);
+                return;
+            }
+        }
+    }
+
+    private static bool IsPerFrameRejection(WorkerFrameProtocolException exception)
+    {
+        return exception.ErrorCode is WorkerFrameProtocolErrorCode.InvalidEnvelope
+            or WorkerFrameProtocolErrorCode.MessageTooLarge
+            or WorkerFrameProtocolErrorCode.ProtocolVersionMismatch
+            or WorkerFrameProtocolErrorCode.SessionMismatch;
+    }
+
+    private PendingFrame? DequeueNext()
+    {
+        lock (_gate)
+        {
+            if (_controlFrames.Count > 0)
+            {
+                return _controlFrames.Dequeue();
+            }
+
+            if (_eventFrames.Count > 0)
+            {
+                return _eventFrames.Dequeue();
+            }
+
+            return null;
+        }
+    }
+
+    private void FailAllQueued(Exception exception)
+    {
+        lock (_gate)
+        {
+            while (_controlFrames.Count > 0)
+            {
+                _controlFrames.Dequeue().Completion.TrySetException(exception);
+            }
+
+            while (_eventFrames.Count > 0)
+            {
+                _eventFrames.Dequeue().Completion.TrySetException(exception);
+            }
+        }
+    }
+
+    private async Task WriteFrameAsync(WorkerEnvelope envelope)
+    {
        WorkerEnvelopeValidator.Validate(envelope, _options);

+        // Stamp the sequence at the actual point of writing, under the write lock, so the wire order
+        // and the stamped sequence agree regardless of caller concurrency or priority (WRK-04).
+        envelope.Sequence = unchecked(++_nextSequence);
+
        int payloadLength = envelope.CalculateSize();
        if (payloadLength == 0)
        {
@@ -55,26 +209,16 @@ public sealed class WorkerFrameWriter
                $"Worker envelope payload length {payloadLength} exceeds the configured maximum of {_options.MaxMessageBytes} bytes.");
        }

-        // Serialize once into a single buffer that carries the 4-byte
-        // length prefix followed by the payload, then issue one stream write.
-        // This avoids a second serialization pass (envelope.ToByteArray()
-        // would re-run CalculateSize internally), a separate prefix array,
-        // and a separate prefix write.
+        // Serialize once into a single buffer that carries the 4-byte length prefix followed by the
+        // payload, then issue one stream write. This avoids a second serialization pass, a separate
+        // prefix array, and a separate prefix write.
        int frameLength = sizeof(uint) + payloadLength;
        byte[] frame = new byte[frameLength];
        WriteUInt32LittleEndian(frame, (uint)payloadLength);
        envelope.WriteTo(new Span<byte>(frame, sizeof(uint), payloadLength));

-        await _writeLock.WaitAsync(cancellationToken).ConfigureAwait(false);
-        try
-        {
-            await _stream.WriteAsync(frame, 0, frameLength, cancellationToken).ConfigureAwait(false);
-            await _stream.FlushAsync(cancellationToken).ConfigureAwait(false);
-        }
-        finally
-        {
-            _writeLock.Release();
-        }
+        await _stream.WriteAsync(frame, 0, frameLength, CancellationToken.None).ConfigureAwait(false);
+        await _stream.FlushAsync(CancellationToken.None).ConfigureAwait(false);
    }

    private static void WriteUInt32LittleEndian(
@@ -18,6 +18,13 @@ public sealed class WorkerPipeSession
    private static readonly TimeSpan BackgroundTaskStopTimeout = TimeSpan.FromSeconds(1);
    private const uint EventDrainBatchSize = 128;

+    // Hard cap on how many events a single DrainEvents diagnostic reply may carry. DrainEvents is a
+    // non-streaming control command, so an unbounded drain (including the max_events = 0 "as many as
+    // available" request) could pack the whole queue into one session-killing reply frame (IPC-04).
+    // The gateway request validator rejects requests above its public ceiling; this worker-side cap is
+    // the backstop and defines the effective per-reply maximum. Kept in step with that public ceiling.
+    private const uint MaxDrainEventsPerReply = 10_000;
+
    private readonly WorkerFrameProtocolOptions _options;
    private readonly Func<int> _processIdProvider;
    private readonly Func<IWorkerRuntimeSession> _runtimeSessionFactory;
@@ -28,7 +35,6 @@ public sealed class WorkerPipeSession
    private readonly object _commandTaskGate = new();
    private readonly HashSet<Task> _activeCommandTasks = new();
    private IWorkerRuntimeSession? _runtimeSession;
-    private long _nextSequence;

    // Mutated from the message loop, command tasks, the heartbeat loop and the
    // shutdown path; volatile so cross-thread reads observe the latest state
@@ -225,6 +231,12 @@ public sealed class WorkerPipeSession
                WorkerFrameProtocolErrorCode.NonceMismatch,
                "GatewayHello nonce does not match the worker launch nonce.");
        }
+
+        // Adopt the gateway-negotiated frame maximum so both ends frame to the same limit instead of
+        // matched compile-time defaults (IPC-02). Applied here, before the message loop, so every
+        // post-handshake frame is validated against the negotiated value; the reader and writer share
+        // this options instance. The hello frame itself was small and already read under the default.
+        _options.AdoptNegotiatedMaxMessageBytes(gatewayHello.MaxFrameBytes);
    }

    private Task WriteWorkerHelloAsync(CancellationToken cancellationToken)
@@ -361,8 +373,11 @@ public sealed class WorkerPipeSession

            foreach (WorkerEvent workerEvent in events)
            {
+                // Events are the low-priority frame class: the writer holds them behind any pending
+                // control frame (reply, fault, heartbeat, shutdown ack) so those are not delayed
+                // behind an event backlog (WRK-07).
                await _writer
-                    .WriteAsync(CreateEnvelope(workerEvent), cancellationToken)
+                    .WriteAsync(CreateEnvelope(workerEvent), WorkerFrameWritePriority.Event, cancellationToken)
                    .ConfigureAwait(false);
            }
        }
@@ -527,7 +542,12 @@ public sealed class WorkerPipeSession
        IWorkerRuntimeSession? runtimeSession = _runtimeSession;
        if (runtimeSession is not null)
        {
-            uint maxEvents = command.DrainEvents?.MaxEvents ?? 0;
+            // Bound the diagnostic drain so max_events = 0 ("as many as available") or an over-large
+            // request cannot pack the whole queue into one session-killing reply frame (IPC-04).
+            uint requested = command.DrainEvents?.MaxEvents ?? 0;
+            uint maxEvents = requested == 0 || requested > MaxDrainEventsPerReply
+                ? MaxDrainEventsPerReply
+                : requested;
            foreach (WorkerEvent workerEvent in runtimeSession.DrainEvents(maxEvents))
            {
                if (workerEvent.Event is not null)
@@ -1004,19 +1024,16 @@ public sealed class WorkerPipeSession

    private WorkerEnvelope CreateBaseEnvelope()
    {
+        // Sequence is deliberately left unset here: the frame writer stamps it at the actual point of
+        // writing, under its single drain task, so the on-wire order and the stamped sequence agree
+        // even under concurrent producers and priority reordering (WRK-04).
        return new WorkerEnvelope
        {
            ProtocolVersion = _options.ProtocolVersion,
            SessionId = _options.SessionId,
-            Sequence = NextSequence(),
        };
    }

-    private ulong NextSequence()
-    {
-        return unchecked((ulong)Interlocked.Increment(ref _nextSequence));
-    }
-
    private async Task<WorkerReady> InitializeMxAccessAsync(CancellationToken cancellationToken)
    {
        // RunAsync constructs the runtime session via _runtimeSessionFactory()
Author	SHA1	Message	Date
Joseph Doherty	de57d834b1	docs(archreview-p1): mark GWC-04 Done — Wave 3 complete ci / portable (push) Failing after 3m27s Details ci / java (push) Failing after 3m30s Details ci / java (pull_request) Failing after 46s Details ci / portable (pull_request) Failing after 51s Details ci / windows (push) Failing after 12m55s Details ci / windows (pull_request) Failing after 38s Details ci / live-mxaccess (push) Has been cancelled Details ci / live-mxaccess (pull_request) Has been cancelled Details	2026-07-09 09:28:39 -04:00
Joseph Doherty	32d6b48910	feat(gateway): decouple worker event enqueue from the read loop (GWC-04) The read loop awaited EnqueueWorkerEventAsync inline, which blocks in the bounded event channel's timed WriteAsync (up to EventChannelFullModeTimeout, default 5s) when the channel is full with no/slow consumer. A WorkerCommandReply or heartbeat queued behind an event frame was then stalled, so an in-flight InvokeAsync could hit CommandTimeout even though the worker replied in time. Mirror the existing outbound WriteLoopAsync: add an unbounded event staging channel and a dedicated EventWriteLoopAsync. DispatchEnvelope is now fully synchronous — the WorkerEvent branch hands the event off with a non-blocking TryWrite and the read loop never awaits. The event write loop owns the timed WriteAsync into the bounded channel and the sustained-overflow ProtocolViolation fault (unchanged contract). Registered in WaitForBackgroundTasks + completed on close/fault/dispose. Test: reply arriving after events with a full, consumer-less event channel is dispatched promptly (no CommandTimeout) — pipe-harness, verified on windev. Docs: GatewayProcessDesign read/write/event-loop section.	2026-07-09 09:28:13 -04:00
Joseph Doherty	bfbfcdd112	docs(archreview-p1): mark IPC-02/03/04 + WRK-04/07 Done (Wave 3, windev-verified)	2026-07-09 09:21:48 -04:00
Joseph Doherty	309296f467	test(ipc): refresh proto descriptor set + MaxMessageBytes default expectation Follow-up to the IPC-02 proto change and IPC-03 headroom default: - regenerate clients/proto/descriptors/mxaccessgw-client-v1.protoset with the pinned protoc 34.1 so it carries GatewayHello.max_frame_bytes (IPC-01 descriptor refresh; ClientProtoInputTests.Descriptor_ContainsEveryContractMessageAndField). - update GatewayOptionsTests design-default assertion to 16 MiB + 64 KiB reserve.	2026-07-09 09:16:31 -04:00
Joseph Doherty	ebe6aeac98	feat(worker): adopt negotiated frame max, bound drain, priority write scheduler (IPC-02/04 + WRK-04/07 worker half) Worker half of the Wave 3 size/backpressure + write-ordering pass: - IPC-02: the worker adopts GatewayHello.max_frame_bytes during the handshake (WorkerFrameProtocolOptions.AdoptNegotiatedMaxMessageBytes) instead of a hard-coded default; 0 keeps the default, a value above a 256 MiB ceiling is rejected. Reader and writer share the options instance, applied before the message loop. - IPC-04: DrainEvents caps each reply at MaxDrainEventsPerReply (10_000) and treats max_events = 0 as that cap rather than 'drain the entire queue', so one diagnostic drain cannot pack a session-killing reply frame. - WRK-04: WorkerFrameWriter stamps the envelope Sequence at the actual point of writing (under the write lock) instead of at envelope creation, so the on-wire order and the stamped sequence always agree under concurrent producers. - WRK-07: the writer is now a cooperative priority scheduler — callers enqueue at Control or Event priority and the draining lock-holder writes all control frames before any event frame, so replies/faults/heartbeats jump ahead of an event backlog. Per-frame validation/size rejections fail only that frame; a stream write failure fails all queued frames. Tests: monotonic gap-free sequence under concurrency, control-before-event priority (gated stream), negotiated-max adoption, DrainEvents zero-bound. Worker builds x86 only — verified on windev.	2026-07-09 09:09:14 -04:00
Joseph Doherty	c8b3a2281a	feat(ipc): negotiate worker frame max, add gRPC headroom, bound DrainEvents (IPC-02/03/04 gateway half) Proto foundation + gateway-side of the size/backpressure pass: - IPC-02: add GatewayHello.max_frame_bytes (regen Generated/); gateway sends its negotiated worker-frame max in the handshake so the worker can adopt it instead of a hard-coded default. Worker read-half lands separately. - IPC-03: give the pipe frame max envelope-overhead headroom above the public gRPC cap (WorkerFrameProtocolOptions.EnvelopeOverheadReserveBytes = 64 KiB; default Worker.MaxMessageBytes bumped to 16 MiB + reserve), cross-validate the headroom at startup, and pre-check command envelope size in WorkerClient so an oversized command fails only that correlation (ResourceExhausted) instead of faulting the whole session. - IPC-04: reject DrainEvents max_events above a public ceiling in the request validator (worker per-reply cap lands with the worker half). Docs: GatewayConfiguration, WorkerFrameProtocol, gateway.md. Tests: headroom validation, DrainEvents bound, oversized-command per-command failure (pipe-harness, verified on windev).	2026-07-09 08:49:59 -04:00
Joseph Doherty	e1a505d662	docs(archreview-p1): record Windows temp-file-lock flakiness fix + windev verification 3 consecutive windev full-suite runs: 793 passed / 2 failed, 0 orphans (was flaky 2-4 fails). The 2 stable failures are pre-existing windev-environmental (cert SAN machine-name assertion; event-stream concurrency timing) and pass on macOS.	2026-07-09 08:20:43 -04:00
Joseph Doherty	11a716a07f	test+fix: eliminate Windows full-suite temp-file-lock flakiness Full-host tests passed in isolation but failed 2-4/run under parallel execution on Windows with 'the process cannot access the file ... because it is being used by another process' (macOS never sees it — Unix deletes open files). Two shared-state parallel collisions, discovered while verifying TST-08: 1. Self-signed cert generation. GatewayTlsBootstrapTests sets process-global Kestrel/TLS env vars that GatewayApplication.Build reads; a parallel host-building test inherits them mid-run and both generate a cert at the same path, racing on the fixed-name '<path>.tmp'. Fixes: - SelfSignedCertificateProvider: stage the PFX in a unique '<path>.<guid>.tmp' instead of a fixed name, so concurrent/interrupted writers never collide on the temp file (real robustness: two instances or a restart-during-write no longer clash). The final atomic Move (last-writer-wins) still yields an equivalent cert. - TestHostEnvironmentInitializer: default MxGateway__Tls__SelfSignedCertPath to a per-process temp path so the suite never writes the shared ProgramData default or fights the deployed service; first host-building test generates, the rest load it. - GatewayTlsBootstrapTests: [Collection] with DisableParallelization so its global env-var mutation cannot bleed into parallel tests (new GlobalEnvironmentCollection). 2. AuthStoreHealthCheckTests opened a real SQLite file; Microsoft.Data.Sqlite's connection pool keeps the .db handle open after 'await using', so the finally File.Delete threw. Reuse the existing TempDatabaseDirectory helper, which calls SqliteConnection.ClearAllPools() before deleting. Server build clean; affected classes 17/17 on macOS. Windows full-suite verified separately.	2026-07-09 08:14:55 -04:00
Joseph Doherty	8c06635ee5	docs(archreview-p1): TST-08 orphaned-testhost leak does not reproduce Empirically verified the full ZB.MOM.WW.MxGateway.Tests suite exits cleanly: - macOS: 0 surviving testhost after a full run. - windev (origin/main worktree): 0 new testhost and 0 new worker processes after a full run, isolated by process StartTime. Static audit confirms the prime-suspect fixtures already dispose deterministically (WorkerClient cancels _stopCts + awaits its read/write/heartbeat tasks with a 5s timeout; GatewaySession disposes the client; harness/E2E fixtures use await using). Drop the stale 'leaves orphaned testhost processes' rationale from CLAUDE.md's Source Update Workflow; keep filtered-run guidance as a speed guideline only. Separately discovered (tracked in 00-tracking.md change log, NOT part of TST-08): Windows-only temp-file-lock flakiness in ~4 full-host test classes (Sqlite connection-pool retaining a temp .db handle; SelfSignedCertificateProvider's fixed-name gw.pfx.tmp racing teardown) that macOS never surfaces.	2026-07-09 08:02:33 -04:00
Joseph Doherty	74e815c4d7	fix(archreview-p1): SEC-02/12/20 dashboard + observability hardening - SEC-02: DashboardAuthorizationHandler restricts the loopback and Authentication:Mode=Disabled bypasses to read-only. They now satisfy only a Viewer-bearing requirement (AnyDashboardRole), never AdminOnly, so anonymous localhost can view the dashboard but cannot reach API-key CRUD or session Close/Kill at the policy layer (previously guarded only by service re-checks). - SEC-12: DashboardSessionAdminService emits canonical AuditEvents through IAuditWriter (actions dashboard-close-session / dashboard-kill-worker, category SessionAdmin) on Success/Failure/Denied, mirroring the API-key audit path so destructive session actions leave durable, queryable rows. - SEC-20: drop the unbounded session_id tag from the exported mxgateway.heartbeats.failed counter (per-session detail stays in the snapshot/log). Docs updated same-change: CLAUDE.md (read-only loopback + 5-min bearer), GatewayDashboardDesign.md (bypass scoping + session-admin audit), Metrics.md. Server build clean; 30/30 targeted + 295/295 Dashboard/Security/App/Metrics sweep.	2026-07-09 07:47:51 -04:00