fix(gateway): remove the worker startup-probe retry options that could never retry (#188)

WorkerProcessStartedProbe only checks HasExited and throws a non-retryable
WorkerProcessLaunchException, so StartupProbeRetryAttempts and
StartupProbeRetryDelayMilliseconds configured a Polly retry that no shipped
probe could trigger. Real readiness is WorkerClient's WorkerReady handshake
under the same StartupTimeoutSeconds budget, so the probe now runs once and
the two options, their validation, the Polly pipeline (and the server's now
unused Polly.Core reference) and their docs are removed. A stale value in
configuration is ignored by the binder.

Tests: WorkerProcessLauncherTests.LaunchAsync_WhenStartupProbeFails_DoesNotRetryTheProbe,
WorkerOptions_HasNoStartupProbeRetryOptions (both fail on the old code).

docs/GatewayConfiguration.md

@@ -22,8 +22,6 @@ paths, timeouts, queue sizes, enum values, or protocol values are invalid.
       "WorkingDirectory": null,
       "RequiredArchitecture": "X86",
       "StartupTimeoutSeconds": 30,
-      "StartupProbeRetryAttempts": 3,
-      "StartupProbeRetryDelayMilliseconds": 250,
       "PipeConnectAttemptTimeoutMilliseconds": 2000,
       "ShutdownTimeoutSeconds": 10,
       "HeartbeatIntervalSeconds": 5,
@@ -116,8 +114,6 @@ launch CWD (SEC-01, SEC-33).
 | `MxGateway:Worker:WorkingDirectory` | `null` | Optional working directory for the worker process. When set, it must be a valid filesystem path. |
 | `MxGateway:Worker:RequiredArchitecture` | `X86` | Required Portable Executable architecture for the worker. Supported values are `X86` and `X64`; MXAccess parity uses `X86`. |
 | `MxGateway:Worker:StartupTimeoutSeconds` | `30` | Total startup budget for process launch, startup probe, pipe connect, handshake, and worker readiness. |
-| `MxGateway:Worker:StartupProbeRetryAttempts` | `3` | Number of retry attempts for transient worker startup probe failures before pipe connection and handshake continue. |
-| `MxGateway:Worker:StartupProbeRetryDelayMilliseconds` | `250` | Delay between transient startup probe retry attempts. |
 | `MxGateway:Worker:PipeConnectAttemptTimeoutMilliseconds` | `2000` | Per-attempt timeout used by the worker named-pipe connect retry path. The overall pipe connection still stays under the startup budget. |
 | `MxGateway:Worker:WriteCompletionWaitMilliseconds` | `1500` | Bounded wait the worker holds a unary write reply (`Write`/`Write2`/`WriteSecured`/`WriteSecured2`; bulk writes excluded) for the matching MXAccess `OnWriteComplete` callback, so the reply's `statuses` carry the real commit outcome. `0` disables the wait (pure fire-and-forget replies). Must be `>= 0`. The gateway conveys the value to the worker via the `MXGATEWAY_WORKER_WRITE_COMPLETION_WAIT_MS` environment variable. Consumers that time their own writes must budget above this wait: OtOpcUa's GalaxyDriver wraps gateway writes in a 2 s Tier A resilience timeout, so a deployment raising this option past ~2000 must raise that driver `ResilienceConfig` write timeout in step or slow-but-successful commits surface as consumer-side failures. |
 | `MxGateway:Worker:EventQueueCapacity` | `10000` | Capacity, in events, of the worker's outbound MXAccess event queue. Must be between `1000` and `1000000`. This is burst headroom, not a throttle: the queue has no drop policy, so filling it records a `QueueOverflow` worker fault and faults the session. Raise it for sessions whose subscription set can outrun the drain loop (large advise sets, slow event consumers); the backing queue pre-allocates its slots, so the ceiling keeps a mistyped value from committing the 32-bit worker to an outsized allocation. The gateway conveys the value to the worker via the `MXGATEWAY_EVENT_QUEUE_CAPACITY` environment variable; a missing or unusable value leaves the worker on the 10000 default rather than failing the session. |
@@ -126,7 +122,6 @@ launch CWD (SEC-01, SEC-33).
 | `MxGateway:Worker:HeartbeatGraceSeconds` | `15` | Maximum age of the last worker heartbeat before the gateway faults the worker. This must be greater than or equal to `HeartbeatIntervalSeconds`. |
 | `MxGateway:Worker:MaxMessageBytes` | `16842752` | Maximum worker IPC frame payload size in bytes. The validator allows values from `1024` through `268435456`, and additionally requires this to be at least `MxGateway:Protocol:MaxGrpcMessageBytes` plus a 65536-byte (64 KiB) envelope-overhead reserve **and at most that cap plus twice the reserve**. The default is the 16 MiB public gRPC cap plus one reserve. The gateway conveys the negotiated value to the worker in the handshake (`GatewayHello.max_frame_bytes`), so the worker frames to the same limit rather than a hard-coded default. The band is narrow by design, so changing either message-size knob alone no longer validates — move `MxGateway:Protocol:MaxGrpcMessageBytes` and this one together, keeping this one one to two reserves above it. |
 
-`StartupProbeRetryAttempts`, `StartupProbeRetryDelayMilliseconds`,
 `PipeConnectAttemptTimeoutMilliseconds`, timeout values, heartbeat values, and
 `MaxMessageBytes` must be positive. `MaxMessageBytes` is intentionally bounded
 to avoid accidental large allocations from malformed or oversized frames. It

docs/GatewayProcessDesign.md

@@ -940,8 +940,6 @@ Suggested configuration shape:
       "WorkingDirectory": null,
       "RequiredArchitecture": "X86",
       "StartupTimeoutSeconds": 30,
-      "StartupProbeRetryAttempts": 3,
-      "StartupProbeRetryDelayMilliseconds": 250,
       "PipeConnectAttemptTimeoutMilliseconds": 2000,
       "ShutdownTimeoutSeconds": 10,
       "HeartbeatIntervalSeconds": 5,

docs/Metrics.md

@@ -109,7 +109,7 @@ The scalar fields mirror the counters and gauges. The four dictionaries provide
 - `CommandFailuresByMethod` keys by gRPC method name.
 - `EventsByFamily` keys by event family (the `Family` enum on a worker event).
 - `EventsBySession` keys by `sessionId`; entries are removed via `RemoveSessionEvents` when a session closes so the map does not grow without bound.
-- `RetryAttemptsByArea` keys by the resilience `area` tag, e.g. `worker_startup`.
+- `RetryAttemptsByArea` keys by the resilience `area` tag passed to `RetryAttempted(area)`. No gateway code path records a retry at present (the worker launcher's startup-probe retry was removed in #188), so it is empty unless a future resilience path records one.
 
 `EventsReceived` is read with `Interlocked.Read(ref _eventsReceived)` because `EventReceived` increments it via `Interlocked.Increment` outside the lock to keep the event-ingestion path non-blocking. `CommandsStarted`, `CommandsSucceeded`, `CommandsFailed`, and `CommandFailuresByMethod` are read the same way: the command counters run two-to-three times per gRPC call, so they are recorded with `Interlocked` and a `ConcurrentDictionary` rather than under `_syncRoot` (PERF-02). The two queue depths are pulled from their registered sources before the lock is taken, since those delegates reach into subscriber channels and worker clients.
 
@@ -152,15 +152,13 @@ _metrics.RemoveSessionEvents(session.SessionId);
 
 ### Worker process launcher
 
-`Workers/WorkerProcessLauncher.cs` records process-level kills and startup retries:
+`Workers/WorkerProcessLauncher.cs` records process-level kills:
 
 ```csharp
 _metrics.WorkerKilled(reason);
-...
-_metrics.RetryAttempted("worker_startup");
 ```
 
-The `worker_startup` tag is hard-coded so the `RetryAttemptsByArea` snapshot reports launcher retries distinctly from other resilience areas.
+The launcher no longer records retries: its startup probe runs once (#188).
 
 ### Session worker client factory
 

docs/WorkerProcessLauncher.md

@@ -49,17 +49,15 @@ Before starting the process, the launcher validates that the configured worker
 path exists, has a `.exe` extension, contains a valid Windows Portable
 Executable header, and matches the configured `RequiredArchitecture`.
 
-After the process starts, `IWorkerStartupProbe` waits for startup readiness.
-The default probe only verifies that the worker did not exit immediately. The
-worker client replaces this probe when pipe connection, hello, and
-`WorkerReady` handling are implemented.
-
-Startup probing uses a bounded Polly retry policy. The gateway starts the worker
-process once, then retries only transient startup-probe failures while the
-process remains alive. The policy is configured by
-`WorkerOptions.StartupProbeRetryAttempts` and
-`WorkerOptions.StartupProbeRetryDelayMilliseconds`; the retry counter is
-recorded as `mxgateway.retries.attempted` with `area=worker_startup`.
+After the process starts, `IWorkerStartupProbe` runs once. The default probe
+only verifies that the worker did not exit immediately; a worker that has
+already exited cannot be recovered by waiting, so the probe is not retried.
+Real readiness is the pipe connection, hello, and `WorkerReady` handshake that
+`WorkerClient` performs after the launch, under the same
+`WorkerOptions.StartupTimeoutSeconds` budget. (The former
+`StartupProbeRetryAttempts` / `StartupProbeRetryDelayMilliseconds` options were
+removed because no shipped probe could ever produce a retryable failure; a
+stale value in configuration is ignored by the binder.)
 
 The worker uses the exported pipe-attempt value inside its own bounded Polly
 retry loop.