Add Polly resilience policies
This commit is contained in:
@@ -37,6 +37,19 @@ The default probe only verifies that the worker did not exit immediately. The
|
||||
worker client replaces this probe when pipe connection, hello, and
|
||||
`WorkerReady` handling are implemented.
|
||||
|
||||
Startup probing uses a bounded Polly retry policy. The gateway starts the worker
|
||||
process once, then retries only transient startup-probe failures while the
|
||||
process remains alive. The policy is configured by
|
||||
`WorkerOptions.StartupProbeRetryAttempts` and
|
||||
`WorkerOptions.StartupProbeRetryDelayMilliseconds`; the retry counter is
|
||||
recorded as `mxgateway.retries.attempted` with `area=worker_startup`.
|
||||
|
||||
The launcher also passes
|
||||
`MXGATEWAY_WORKER_PIPE_CONNECT_ATTEMPT_TIMEOUT_MS` to the worker process from
|
||||
`WorkerOptions.PipeConnectAttemptTimeoutMilliseconds`. The worker uses that
|
||||
value as the per-attempt named-pipe connect timeout inside its own bounded
|
||||
Polly retry loop.
|
||||
|
||||
If startup fails or exceeds `WorkerOptions.StartupTimeoutSeconds`, the launcher
|
||||
kills the worker process tree, disposes the process handle, disposes the
|
||||
optional pipe reservation, records a worker kill metric, and reports a
|
||||
|
||||
@@ -110,10 +110,17 @@ public sealed class MxGatewayClientOptions
|
||||
public string? ServerNameOverride { get; init; }
|
||||
public TimeSpan ConnectTimeout { get; init; } = TimeSpan.FromSeconds(10);
|
||||
public TimeSpan DefaultCallTimeout { get; init; } = TimeSpan.FromSeconds(30);
|
||||
public MxGatewayClientRetryOptions Retry { get; init; } = new();
|
||||
public ILoggerFactory? LoggerFactory { get; init; }
|
||||
}
|
||||
```
|
||||
|
||||
The .NET client applies a bounded Polly retry policy only to idempotent calls:
|
||||
`CloseSession` and diagnostic `Invoke` commands such as `Ping`,
|
||||
`GetSessionState`, and `GetWorkerInfo`. It does not retry `OpenSession`, event
|
||||
streams, writes, secured writes, authentication, registration, item management,
|
||||
or subscription changes because those calls can partially succeed in MXAccess.
|
||||
|
||||
API key may be loaded from `MXGATEWAY_API_KEY` by the CLI, not implicitly by the
|
||||
library constructor unless a helper explicitly says it does that.
|
||||
|
||||
|
||||
@@ -843,6 +843,9 @@ Suggested configuration shape:
|
||||
"Worker": {
|
||||
"ExecutablePath": "src/MxGateway.Worker/bin/x86/Release/MxGateway.Worker.exe",
|
||||
"StartupTimeoutSeconds": 30,
|
||||
"StartupProbeRetryAttempts": 3,
|
||||
"StartupProbeRetryDelayMilliseconds": 250,
|
||||
"PipeConnectAttemptTimeoutMilliseconds": 2000,
|
||||
"ShutdownTimeoutSeconds": 10,
|
||||
"HeartbeatIntervalSeconds": 5,
|
||||
"HeartbeatGraceSeconds": 15,
|
||||
|
||||
Reference in New Issue
Block a user