feat(worker): log exception messages and per-command start/end (mxaccessgw#184)
R21-O3: WorkerApplication.Run's failure lines logged only the exception type. Each now also carries exception_message, scrubbed by the new WorkerLogRedactor.RedactMessage: the value of any name=value / name: value pair whose name looks sensitive and any mxgw_ API key token are replaced, line breaks are flattened and the text is capped at 512 characters. The AppDomain unhandled-exception hook stays type-only, since an exception from an arbitrary thread can carry command values. R21-O7: StaCommandDispatcher.ExecuteQueuedCommandAsync logs WorkerCommandStarted (correlation id, method, queued_ms) and WorkerCommandCompleted (status, HRESULT, elapsed_ms) when a CommandLogger is set. IWorkerLogger has no Debug level, so the lines are opt-in through MXGATEWAY_WORKER_LOG_COMMANDS=1|true (WorkerCommandLogging), which the worker inherits from the gateway process environment. No payloads or diagnostic messages are logged. Docs updated. (mxaccessgw#184) Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
@@ -124,7 +124,7 @@ public static WorkerBootstrapResult Failure(WorkerExitCode exitCode, IEnumerable
|
||||
|
||||
### Unhandled-exception hook
|
||||
|
||||
The production entry point, `WorkerApplication.Run(string[])`, subscribes `AppDomain.CurrentDomain.UnhandledException` before it parses arguments. The handler (`WorkerApplication.HandleUnhandledException`) logs one `WorkerUnhandledException` error line and calls `Environment.Exit(7)`. Without it, an exception on a thread that `Run` does not own ended the process with the CLR's crash code and no worker log line, so the gateway saw an unexplained exit. The line carries `exit_code`, `exception_type`, and `is_terminating` only: the exception message and stack are left out because they can carry tag values or credentials passed through a command, the same rule every other worker failure line follows. A failure inside the logger is swallowed so the exit always happens. The injectable `Run` overloads that tests call do not install the hook, because it is process-global.
|
||||
The production entry point, `WorkerApplication.Run(string[])`, subscribes `AppDomain.CurrentDomain.UnhandledException` before it parses arguments. The handler (`WorkerApplication.HandleUnhandledException`) logs one `WorkerUnhandledException` error line and calls `Environment.Exit(7)`. Without it, an exception on a thread that `Run` does not own ended the process with the CLR's crash code and no worker log line, so the gateway saw an unexplained exit. The line carries `exit_code`, `exception_type`, and `is_terminating` only: the exception message and stack are left out because an exception from an arbitrary thread (a COM callback, a command path) can carry tag values or credentials passed through a command. A failure inside the logger is swallowed so the exit always happens. The injectable `Run` overloads that tests call do not install the hook, because it is process-global.
|
||||
|
||||
## Environment Abstraction
|
||||
|
||||
@@ -181,6 +181,12 @@ private void Write(
|
||||
}
|
||||
```
|
||||
|
||||
### Per-command start/end lines
|
||||
|
||||
`StaCommandDispatcher` can log one `WorkerCommandStarted` and one `WorkerCommandCompleted` line per STA command. Both carry `session_id`, `correlation_id`, and `command_method`; the start line adds `queued_ms` (time since the gateway enqueued the command) and the end line adds `protocol_status`, `hresult`, and `elapsed_ms` (time on the STA). Neither carries the command payload or the reply's diagnostic message, so tag values, secured-write credentials and provider detail stay out of the log.
|
||||
|
||||
The lines are off by default because a busy session dispatches many commands and `IWorkerLogger` has no Debug level to put them behind. Setting the environment variable `MXGATEWAY_WORKER_LOG_COMMANDS` to `1` or `true` turns them on (`WorkerCommandLogging`). The gateway launcher does not set it; a worker inherits it from the gateway process environment, so setting it on the gateway service enables it for every worker that service launches.
|
||||
|
||||
### What the redactor redacts and why
|
||||
|
||||
`gateway.md` "Security" requires that the worker never log raw credential values for `AuthenticateUser`, `WriteSecured`, or related secured operations. The bootstrap nonce is also a credential: anyone who reads it can impersonate the worker to the gateway pipe. `WorkerLogRedactor` enforces this by replacing values whose field name contains any of these substrings (case-insensitive) with the literal `[redacted]`:
|
||||
@@ -220,7 +226,7 @@ if (!result.Succeeded)
|
||||
}
|
||||
```
|
||||
|
||||
On success it logs `WorkerBootstrapSucceeded` with the session fields (the `nonce` field is redacted by `WorkerLogRedactor` because of its name), hands the `WorkerOptions` to `IWorkerPipeClient.RunAsync`, and waits synchronously. The `try/catch` around the pipe call maps a `WorkerFrameProtocolException` with `EndOfStream` to `PipeConnectionFailed` (a closed pipe is a lost connection, not a malformed frame), any other `WorkerFrameProtocolException` to `ProtocolViolation`, a `WorkerEventQueueFaultException` with category `QueueOverflow` to `EventQueueOverflow`, `IOException`/`TimeoutException` to `PipeConnectionFailed`, and any other exception to `UnexpectedFailure`, so every code path through `Run` returns one of the values in `WorkerExitCode`.
|
||||
On success it logs `WorkerBootstrapSucceeded` with the session fields (the `nonce` field is redacted by `WorkerLogRedactor` because of its name), hands the `WorkerOptions` to `IWorkerPipeClient.RunAsync`, and waits synchronously. The `try/catch` around the pipe call maps a `WorkerFrameProtocolException` with `EndOfStream` to `PipeConnectionFailed` (a closed pipe is a lost connection, not a malformed frame), any other `WorkerFrameProtocolException` to `ProtocolViolation`, a `WorkerEventQueueFaultException` with category `QueueOverflow` to `EventQueueOverflow`, `IOException`/`TimeoutException` to `PipeConnectionFailed`, and any other exception to `UnexpectedFailure`, so every code path through `Run` returns one of the values in `WorkerExitCode`. Each of those failure lines carries `exit_code`, `exception_type`, and `exception_message`. The message is passed through `WorkerLogRedactor.RedactMessage` first, which scrubs the value of any `name=value` or `name: value` pair whose name looks sensitive (nonce, secret, password, token, credential, API key) and any `mxgw_` API key token, flattens line breaks, and caps the text at 512 characters. Field-name redaction alone cannot see inside free text.
|
||||
|
||||
## Related Documentation
|
||||
|
||||
|
||||
Reference in New Issue
Block a user