fix(worker): exit 5 on gateway death, dedicated exit 8 on event-queue overflow (mxaccessgw#186)
A frame read that hits end of stream means the gateway closed its end of the pipe (usually because it died). WorkerApplication mapped every WorkerFrameProtocolException to ProtocolViolation (6); EndOfStream now exits as PipeConnectionFailed (5), alongside the IOException a broken pipe raises. Event-queue overflow exited as the generic UnexpectedFailure (1). The event drain loop now unwinds with WorkerEventQueueFaultException carrying the fault category, and QueueOverflow exits with the new WorkerExitCode.EventQueueOverflow (8). Other categories keep exit 1. The gateway does not decode worker exit codes (it only logs the number), so no gateway change is needed. WorkerBootstrap.md and MxAccessWorkerInstanceDesign.md updated. (mxaccessgw#186) Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
@@ -133,9 +133,10 @@ starts the STA, or touches MXAccess:
|
|||||||
| `2` | `InvalidArguments` | Required arguments are missing or unknown arguments are present. |
|
| `2` | `InvalidArguments` | Required arguments are missing or unknown arguments are present. |
|
||||||
| `3` | `InvalidProtocolVersion` | `--protocol-version` is not numeric or does not match the supported worker protocol. |
|
| `3` | `InvalidProtocolVersion` | `--protocol-version` is not numeric or does not match the supported worker protocol. |
|
||||||
| `4` | `MissingNonce` | `MXGATEWAY_WORKER_NONCE` is absent or empty. |
|
| `4` | `MissingNonce` | `MXGATEWAY_WORKER_NONCE` is absent or empty. |
|
||||||
| `5` | `PipeConnectionFailed` | An `IOException` or `TimeoutException` escapes the pipe client. |
|
| `5` | `PipeConnectionFailed` | An `IOException` or `TimeoutException` escapes the pipe client, or a frame read hits end of stream because the gateway closed the pipe (gateway death). |
|
||||||
| `6` | `ProtocolViolation` | A `WorkerFrameProtocolException` escapes the pipe client. |
|
| `6` | `ProtocolViolation` | Any other `WorkerFrameProtocolException` escapes the pipe client. |
|
||||||
| `7` | `UnhandledException` | An exception reaches the `AppDomain.UnhandledException` hook (see `docs/WorkerBootstrap.md`). |
|
| `7` | `UnhandledException` | An exception reaches the `AppDomain.UnhandledException` hook (see `docs/WorkerBootstrap.md`). |
|
||||||
|
| `8` | `EventQueueOverflow` | The outbound MXAccess event queue reached capacity (see the event overflow policy below). |
|
||||||
|
|
||||||
Bootstrap logs use `WorkerConsoleLogger` key/value output. `WorkerLogRedactor`
|
Bootstrap logs use `WorkerConsoleLogger` key/value output. `WorkerLogRedactor`
|
||||||
redacts fields whose names indicate nonce, secret, password, token,
|
redacts fields whose names indicate nonce, secret, password, token,
|
||||||
@@ -868,9 +869,10 @@ category `QueueOverflow`. The event drain loop's next pass observes the fault
|
|||||||
through `DrainFault()`, writes the fault frame — a Control-priority frame, so
|
through `DrainFault()`, writes the fault frame — a Control-priority frame, so
|
||||||
it is not stuck behind a queued event backlog — and then throws to unwind
|
it is not stuck behind a queued event backlog — and then throws to unwind
|
||||||
`RunAsync`: the worker process exits rather than continuing in a state where
|
`RunAsync`: the worker process exits rather than continuing in a state where
|
||||||
events are being silently lost. The exit currently uses the generic
|
events are being silently lost. The drain loop unwinds with a
|
||||||
`WorkerExitCode.UnexpectedFailure` rather than a dedicated overflow code; a
|
`WorkerEventQueueFaultException` carrying the fault category, and the process
|
||||||
distinct exit code remains open (tracked separately). Do not drop or coalesce
|
boundary maps `QueueOverflow` to the dedicated `WorkerExitCode.EventQueueOverflow`
|
||||||
|
(8), so an overflow is identifiable from the worker's exit code alone. Do not drop or coalesce
|
||||||
events to avoid this exit — that is explicitly out of scope for v1. Production
|
events to avoid this exit — that is explicitly out of scope for v1. Production
|
||||||
coalescing may be added later, but it must be explicit and tested.
|
coalescing may be added later, but it must be explicit and tested.
|
||||||
|
|
||||||
|
|||||||
@@ -115,9 +115,10 @@ public static WorkerBootstrapResult Failure(WorkerExitCode exitCode, IEnumerable
|
|||||||
| `InvalidArguments` | 2 | One or more `--session-id`, `--pipe-name`, or `--protocol-version` errors (missing, empty, unknown flag, or no value). |
|
| `InvalidArguments` | 2 | One or more `--session-id`, `--pipe-name`, or `--protocol-version` errors (missing, empty, unknown flag, or no value). |
|
||||||
| `InvalidProtocolVersion` | 3 | `--protocol-version` is not a `uint` or does not equal `GatewayContractInfo.WorkerProtocolVersion`. |
|
| `InvalidProtocolVersion` | 3 | `--protocol-version` is not a `uint` or does not equal `GatewayContractInfo.WorkerProtocolVersion`. |
|
||||||
| `MissingNonce` | 4 | The `MXGATEWAY_WORKER_NONCE` environment variable is null, empty, or whitespace. |
|
| `MissingNonce` | 4 | The `MXGATEWAY_WORKER_NONCE` environment variable is null, empty, or whitespace. |
|
||||||
| `PipeConnectionFailed` | 5 | An `IOException` or `TimeoutException` escapes the pipe client. |
|
| `PipeConnectionFailed` | 5 | An `IOException` or `TimeoutException` escapes the pipe client, or a `WorkerFrameProtocolException` with `EndOfStream` does — the gateway closed its end of the pipe, usually because it died. |
|
||||||
| `ProtocolViolation` | 6 | A `WorkerFrameProtocolException` escapes the pipe client. |
|
| `ProtocolViolation` | 6 | Any other `WorkerFrameProtocolException` escapes the pipe client. |
|
||||||
| `UnhandledException` | 7 | An exception reaches `AppDomain.CurrentDomain.UnhandledException` — thrown on a thread outside `WorkerApplication.Run`'s `try/catch` (a background task continuation, a timer, a COM callback thread). |
|
| `UnhandledException` | 7 | An exception reaches `AppDomain.CurrentDomain.UnhandledException` — thrown on a thread outside `WorkerApplication.Run`'s `try/catch` (a background task continuation, a timer, a COM callback thread). |
|
||||||
|
| `EventQueueOverflow` | 8 | The outbound MXAccess event queue reached capacity: the event drain loop wrote the `QueueOverflow` fault and unwound with a `WorkerEventQueueFaultException`. Other event-queue fault categories still exit as `UnexpectedFailure`. |
|
||||||
|
|
||||||
`InvalidArguments`, `InvalidProtocolVersion`, and `MissingNonce` originate in the parser; `UnhandledException` originates in the process-wide hook described below; the others originate in `WorkerApplication.Run`'s `try/catch` around the pipe client.
|
`InvalidArguments`, `InvalidProtocolVersion`, and `MissingNonce` originate in the parser; `UnhandledException` originates in the process-wide hook described below; the others originate in `WorkerApplication.Run`'s `try/catch` around the pipe client.
|
||||||
|
|
||||||
@@ -219,7 +220,7 @@ if (!result.Succeeded)
|
|||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
On success it logs `WorkerBootstrapSucceeded` with the session fields (the `nonce` field is redacted by `WorkerLogRedactor` because of its name), hands the `WorkerOptions` to `IWorkerPipeClient.RunAsync`, and waits synchronously. The `try/catch` around the pipe call maps `WorkerFrameProtocolException` to `ProtocolViolation`, `IOException`/`TimeoutException` to `PipeConnectionFailed`, and any other exception to `UnexpectedFailure`, so every code path through `Run` returns one of the values in `WorkerExitCode`.
|
On success it logs `WorkerBootstrapSucceeded` with the session fields (the `nonce` field is redacted by `WorkerLogRedactor` because of its name), hands the `WorkerOptions` to `IWorkerPipeClient.RunAsync`, and waits synchronously. The `try/catch` around the pipe call maps a `WorkerFrameProtocolException` with `EndOfStream` to `PipeConnectionFailed` (a closed pipe is a lost connection, not a malformed frame), any other `WorkerFrameProtocolException` to `ProtocolViolation`, a `WorkerEventQueueFaultException` with category `QueueOverflow` to `EventQueueOverflow`, `IOException`/`TimeoutException` to `PipeConnectionFailed`, and any other exception to `UnexpectedFailure`, so every code path through `Run` returns one of the values in `WorkerExitCode`.
|
||||||
|
|
||||||
## Related Documentation
|
## Related Documentation
|
||||||
|
|
||||||
|
|||||||
@@ -3,6 +3,7 @@ using System.Collections.Generic;
|
|||||||
using System.Threading;
|
using System.Threading;
|
||||||
using System.Threading.Tasks;
|
using System.Threading.Tasks;
|
||||||
using ZB.MOM.WW.MxGateway.Contracts;
|
using ZB.MOM.WW.MxGateway.Contracts;
|
||||||
|
using ZB.MOM.WW.MxGateway.Contracts.Proto;
|
||||||
using ZB.MOM.WW.MxGateway.Worker.Bootstrap;
|
using ZB.MOM.WW.MxGateway.Worker.Bootstrap;
|
||||||
using ZB.MOM.WW.MxGateway.Worker.Ipc;
|
using ZB.MOM.WW.MxGateway.Worker.Ipc;
|
||||||
|
|
||||||
@@ -103,6 +104,81 @@ public sealed class WorkerApplicationTests
|
|||||||
Assert.Equal("WorkerPipeProtocolFailure", logger.Entries[1].EventName);
|
Assert.Equal("WorkerPipeProtocolFailure", logger.Entries[1].EventName);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// A frame read that hits end of stream means the gateway closed the pipe (typically it
|
||||||
|
/// died). That is a lost connection, so it exits as <c>PipeConnectionFailed</c> (5), not
|
||||||
|
/// <c>ProtocolViolation</c> (6) (mxaccessgw#186).
|
||||||
|
/// </summary>
|
||||||
|
[Fact]
|
||||||
|
public void Run_WhenGatewayClosesPipe_ReturnsPipeConnectionFailed()
|
||||||
|
{
|
||||||
|
MemoryWorkerEnvironment environment = CreateEnvironment("nonce-secret");
|
||||||
|
MemoryWorkerLogger logger = new();
|
||||||
|
|
||||||
|
int exitCode = ZB.MOM.WW.MxGateway.Worker.WorkerApplication.Run(
|
||||||
|
ValidArgs(),
|
||||||
|
environment,
|
||||||
|
logger,
|
||||||
|
new ThrowingPipeClient(new WorkerFrameProtocolException(
|
||||||
|
WorkerFrameProtocolErrorCode.EndOfStream,
|
||||||
|
"Worker frame ended before the expected number of bytes were read.")));
|
||||||
|
|
||||||
|
Assert.Equal((int)WorkerExitCode.PipeConnectionFailed, exitCode);
|
||||||
|
Assert.Equal(5, exitCode);
|
||||||
|
MemoryWorkerLogEntry entry = logger.Entries[1];
|
||||||
|
Assert.Equal("WorkerPipeConnectionFailed", entry.EventName);
|
||||||
|
Assert.Equal(WorkerExitCode.PipeConnectionFailed, entry.Fields["exit_code"]);
|
||||||
|
Assert.Equal(WorkerFrameProtocolErrorCode.EndOfStream, entry.Fields["error_code"]);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Event-queue overflow exits with its own code (8) instead of the generic
|
||||||
|
/// <c>UnexpectedFailure</c> (1), so an overflow is identifiable from the exit code alone
|
||||||
|
/// (mxaccessgw#186).
|
||||||
|
/// </summary>
|
||||||
|
[Fact]
|
||||||
|
public void Run_WhenEventQueueOverflows_ReturnsEventQueueOverflow()
|
||||||
|
{
|
||||||
|
MemoryWorkerEnvironment environment = CreateEnvironment("nonce-secret");
|
||||||
|
MemoryWorkerLogger logger = new();
|
||||||
|
|
||||||
|
int exitCode = ZB.MOM.WW.MxGateway.Worker.WorkerApplication.Run(
|
||||||
|
ValidArgs(),
|
||||||
|
environment,
|
||||||
|
logger,
|
||||||
|
new ThrowingPipeClient(new WorkerEventQueueFaultException(
|
||||||
|
WorkerFaultCategory.QueueOverflow,
|
||||||
|
"MXAccess outbound event queue reached capacity 4.")));
|
||||||
|
|
||||||
|
Assert.Equal((int)WorkerExitCode.EventQueueOverflow, exitCode);
|
||||||
|
Assert.Equal(8, exitCode);
|
||||||
|
MemoryWorkerLogEntry entry = logger.Entries[1];
|
||||||
|
Assert.Equal("Error", entry.Level);
|
||||||
|
Assert.Equal("WorkerEventQueueOverflow", entry.EventName);
|
||||||
|
Assert.Equal(WorkerExitCode.EventQueueOverflow, entry.Fields["exit_code"]);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// An event-queue fault of any other category keeps the generic exit code; only overflow
|
||||||
|
/// has a dedicated one.
|
||||||
|
/// </summary>
|
||||||
|
[Fact]
|
||||||
|
public void Run_WhenEventQueueFaultsForAnotherReason_ReturnsUnexpectedFailure()
|
||||||
|
{
|
||||||
|
MemoryWorkerEnvironment environment = CreateEnvironment("nonce-secret");
|
||||||
|
MemoryWorkerLogger logger = new();
|
||||||
|
|
||||||
|
int exitCode = ZB.MOM.WW.MxGateway.Worker.WorkerApplication.Run(
|
||||||
|
ValidArgs(),
|
||||||
|
environment,
|
||||||
|
logger,
|
||||||
|
new ThrowingPipeClient(new WorkerEventQueueFaultException(
|
||||||
|
WorkerFaultCategory.MxaccessEventConversionFailed,
|
||||||
|
"conversion failed")));
|
||||||
|
|
||||||
|
Assert.Equal((int)WorkerExitCode.UnexpectedFailure, exitCode);
|
||||||
|
}
|
||||||
|
|
||||||
/// <summary>Verifies that unexpected exceptions during bootstrap are logged.</summary>
|
/// <summary>Verifies that unexpected exceptions during bootstrap are logged.</summary>
|
||||||
[Fact]
|
[Fact]
|
||||||
public void Run_WithUnexpectedBootstrapFailure_ReturnsUnexpectedFailure()
|
public void Run_WithUnexpectedBootstrapFailure_ReturnsUnexpectedFailure()
|
||||||
|
|||||||
@@ -1998,6 +1998,47 @@ public sealed class WorkerPipeSessionTests
|
|||||||
await SendShutdownAndWaitAsync(pipePair, runTask, cancellation.Token);
|
await SendShutdownAndWaitAsync(pipePair, runTask, cancellation.Token);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// When the event queue records an overflow fault, the drain loop writes the fault frame and
|
||||||
|
/// unwinds with <see cref="WorkerEventQueueFaultException"/> carrying the category, which is
|
||||||
|
/// what lets the process boundary exit with the dedicated overflow code (mxaccessgw#186).
|
||||||
|
/// </summary>
|
||||||
|
/// <returns>A task that represents the asynchronous operation.</returns>
|
||||||
|
[Fact]
|
||||||
|
public async Task RunAsync_WhenEventQueueOverflows_WritesFaultAndThrowsTypedException()
|
||||||
|
{
|
||||||
|
using CancellationTokenSource cancellation = new(TimeSpan.FromSeconds(10));
|
||||||
|
using PipePair pipePair = await PipePair.CreateAsync(cancellation.Token);
|
||||||
|
FakeRuntimeSession runtime = new()
|
||||||
|
{
|
||||||
|
PendingFault = new WorkerFault
|
||||||
|
{
|
||||||
|
Category = WorkerFaultCategory.QueueOverflow,
|
||||||
|
DiagnosticMessage = "MXAccess outbound event queue reached capacity 4.",
|
||||||
|
ProtocolStatus = new ProtocolStatus
|
||||||
|
{
|
||||||
|
Code = ProtocolStatusCode.WorkerUnavailable,
|
||||||
|
Message = "MXAccess outbound event queue reached capacity 4.",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
};
|
||||||
|
WorkerPipeSession session = CreatePipeSession(pipePair.WorkerStream, runtime);
|
||||||
|
Task runTask = session.RunAsync(cancellation.Token);
|
||||||
|
await CompleteGatewayHandshakeAsync(pipePair, cancellation.Token);
|
||||||
|
|
||||||
|
WorkerEnvelope faultEnvelope = await ReadUntilAsync(
|
||||||
|
pipePair.GatewayReader,
|
||||||
|
WorkerEnvelope.BodyOneofCase.WorkerFault,
|
||||||
|
cancellation.Token);
|
||||||
|
Assert.Equal(WorkerFaultCategory.QueueOverflow, faultEnvelope.WorkerFault.Category);
|
||||||
|
|
||||||
|
Task completedTask = await Task.WhenAny(runTask, Task.Delay(TimeSpan.FromSeconds(5), cancellation.Token));
|
||||||
|
Assert.Same(runTask, completedTask);
|
||||||
|
WorkerEventQueueFaultException exception =
|
||||||
|
await Assert.ThrowsAsync<WorkerEventQueueFaultException>(async () => await runTask);
|
||||||
|
Assert.Equal(WorkerFaultCategory.QueueOverflow, exception.Category);
|
||||||
|
}
|
||||||
|
|
||||||
private static WorkerPipeSession CreateSession(
|
private static WorkerPipeSession CreateSession(
|
||||||
Stream inbound,
|
Stream inbound,
|
||||||
Stream outbound,
|
Stream outbound,
|
||||||
|
|||||||
@@ -380,10 +380,21 @@ internal sealed class FakeRuntimeSession : IWorkerRuntimeSession
|
|||||||
return eventSignal.WaitAsync(timeout, cancellationToken);
|
return eventSignal.WaitAsync(timeout, cancellationToken);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Gets or sets a fault the next <see cref="DrainFault"/> call hands back once, standing in
|
||||||
|
/// for the fault a real event queue self-records on overflow.
|
||||||
|
/// </summary>
|
||||||
|
public WorkerFault? PendingFault { get; set; }
|
||||||
|
|
||||||
/// <inheritdoc />
|
/// <inheritdoc />
|
||||||
public WorkerFault? DrainFault()
|
public WorkerFault? DrainFault()
|
||||||
{
|
{
|
||||||
return null;
|
lock (gate)
|
||||||
|
{
|
||||||
|
WorkerFault? fault = PendingFault;
|
||||||
|
PendingFault = null;
|
||||||
|
return fault;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// <summary>
|
/// <summary>
|
||||||
|
|||||||
@@ -7,6 +7,11 @@ public enum WorkerExitCode
|
|||||||
InvalidArguments = 2,
|
InvalidArguments = 2,
|
||||||
InvalidProtocolVersion = 3,
|
InvalidProtocolVersion = 3,
|
||||||
MissingNonce = 4,
|
MissingNonce = 4,
|
||||||
|
/// <summary>
|
||||||
|
/// The pipe to the gateway could not be connected, or it closed under the worker — an
|
||||||
|
/// <c>IOException</c>, a connect <c>TimeoutException</c>, or a frame read that hit end of
|
||||||
|
/// stream because the gateway died or closed its end.
|
||||||
|
/// </summary>
|
||||||
PipeConnectionFailed = 5,
|
PipeConnectionFailed = 5,
|
||||||
ProtocolViolation = 6,
|
ProtocolViolation = 6,
|
||||||
|
|
||||||
@@ -18,4 +23,10 @@ public enum WorkerExitCode
|
|||||||
/// exit code.
|
/// exit code.
|
||||||
/// </summary>
|
/// </summary>
|
||||||
UnhandledException = 7,
|
UnhandledException = 7,
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// The outbound MXAccess event queue reached capacity. The worker wrote a
|
||||||
|
/// <c>QueueOverflow</c> fault and exited rather than drop events (fail-fast backpressure).
|
||||||
|
/// </summary>
|
||||||
|
EventQueueOverflow = 8,
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,25 @@
|
|||||||
|
using System;
|
||||||
|
using ZB.MOM.WW.MxGateway.Contracts.Proto;
|
||||||
|
|
||||||
|
namespace ZB.MOM.WW.MxGateway.Worker.Ipc;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Raised by the event drain loop after it has written the fault the MXAccess event queue
|
||||||
|
/// recorded, to unwind the pipe session. It carries the fault category so the process boundary
|
||||||
|
/// can pick an exit code: <see cref="WorkerFaultCategory.QueueOverflow"/> exits as
|
||||||
|
/// <c>WorkerExitCode.EventQueueOverflow</c> rather than the generic unexpected-failure code.
|
||||||
|
/// </summary>
|
||||||
|
public sealed class WorkerEventQueueFaultException : Exception
|
||||||
|
{
|
||||||
|
/// <summary>Initializes a new instance of <see cref="WorkerEventQueueFaultException"/>.</summary>
|
||||||
|
/// <param name="category">Category of the fault the event queue recorded.</param>
|
||||||
|
/// <param name="message">Diagnostic message.</param>
|
||||||
|
public WorkerEventQueueFaultException(WorkerFaultCategory category, string message)
|
||||||
|
: base(message)
|
||||||
|
{
|
||||||
|
Category = category;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>Gets the category of the fault the event queue recorded.</summary>
|
||||||
|
public WorkerFaultCategory Category { get; }
|
||||||
|
}
|
||||||
@@ -592,7 +592,8 @@ public sealed class WorkerPipeSession
|
|||||||
{
|
{
|
||||||
_state = WorkerState.Faulted;
|
_state = WorkerState.Faulted;
|
||||||
await TryWriteFaultAsync(fault, cancellationToken).ConfigureAwait(false);
|
await TryWriteFaultAsync(fault, cancellationToken).ConfigureAwait(false);
|
||||||
throw new InvalidOperationException(
|
throw new WorkerEventQueueFaultException(
|
||||||
|
fault.Category,
|
||||||
string.IsNullOrWhiteSpace(fault.DiagnosticMessage)
|
string.IsNullOrWhiteSpace(fault.DiagnosticMessage)
|
||||||
? $"MXAccess event queue faulted with category {fault.Category}."
|
? $"MXAccess event queue faulted with category {fault.Category}."
|
||||||
: fault.DiagnosticMessage);
|
: fault.DiagnosticMessage);
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
using System;
|
using System;
|
||||||
using System.Collections.Generic;
|
using System.Collections.Generic;
|
||||||
using System.IO;
|
using System.IO;
|
||||||
|
using ZB.MOM.WW.MxGateway.Contracts.Proto;
|
||||||
using ZB.MOM.WW.MxGateway.Worker.Bootstrap;
|
using ZB.MOM.WW.MxGateway.Worker.Bootstrap;
|
||||||
using ZB.MOM.WW.MxGateway.Worker.Ipc;
|
using ZB.MOM.WW.MxGateway.Worker.Ipc;
|
||||||
|
|
||||||
@@ -158,6 +159,22 @@ public static class WorkerApplication
|
|||||||
|
|
||||||
return (int)WorkerExitCode.Success;
|
return (int)WorkerExitCode.Success;
|
||||||
}
|
}
|
||||||
|
catch (WorkerFrameProtocolException exception)
|
||||||
|
when (exception.ErrorCode == WorkerFrameProtocolErrorCode.EndOfStream)
|
||||||
|
{
|
||||||
|
// A read that hits end of stream means the gateway closed its end of the pipe — most
|
||||||
|
// often because the gateway process died. That is a lost connection, not a malformed
|
||||||
|
// frame, so it exits as PipeConnectionFailed alongside the IOException a broken pipe
|
||||||
|
// raises, rather than as ProtocolViolation.
|
||||||
|
logger.Error("WorkerPipeConnectionFailed", new Dictionary<string, object?>
|
||||||
|
{
|
||||||
|
["exit_code"] = WorkerExitCode.PipeConnectionFailed,
|
||||||
|
["error_code"] = exception.ErrorCode,
|
||||||
|
["exception_type"] = exception.GetType().FullName,
|
||||||
|
});
|
||||||
|
|
||||||
|
return (int)WorkerExitCode.PipeConnectionFailed;
|
||||||
|
}
|
||||||
catch (WorkerFrameProtocolException exception)
|
catch (WorkerFrameProtocolException exception)
|
||||||
{
|
{
|
||||||
logger.Error("WorkerPipeProtocolFailure", new Dictionary<string, object?>
|
logger.Error("WorkerPipeProtocolFailure", new Dictionary<string, object?>
|
||||||
@@ -169,6 +186,17 @@ public static class WorkerApplication
|
|||||||
|
|
||||||
return (int)WorkerExitCode.ProtocolViolation;
|
return (int)WorkerExitCode.ProtocolViolation;
|
||||||
}
|
}
|
||||||
|
catch (WorkerEventQueueFaultException exception)
|
||||||
|
when (exception.Category == WorkerFaultCategory.QueueOverflow)
|
||||||
|
{
|
||||||
|
logger.Error("WorkerEventQueueOverflow", new Dictionary<string, object?>
|
||||||
|
{
|
||||||
|
["exit_code"] = WorkerExitCode.EventQueueOverflow,
|
||||||
|
["exception_type"] = exception.GetType().FullName,
|
||||||
|
});
|
||||||
|
|
||||||
|
return (int)WorkerExitCode.EventQueueOverflow;
|
||||||
|
}
|
||||||
catch (Exception exception) when (exception is IOException or TimeoutException)
|
catch (Exception exception) when (exception is IOException or TimeoutException)
|
||||||
{
|
{
|
||||||
logger.Error("WorkerPipeConnectionFailed", new Dictionary<string, object?>
|
logger.Error("WorkerPipeConnectionFailed", new Dictionary<string, object?>
|
||||||
|
|||||||
Reference in New Issue
Block a user