fix(grpc): record the real StreamEvents disconnect reason

StreamDisconnected always recorded "Detached". The reason tag is now a
bounded EventStreamDisconnectReasons constant for how the stream ended:
SessionClosed, Overflow, WorkerFault, ClientCancelled, Error, or Detached.
This commit is contained in:
Joseph Doherty
2026-10-22 06:29:31 -04:00
parent 0b4a1682c8
commit 6ae5ed905f
5 changed files with 260 additions and 5 deletions
+2 -2
View File
@@ -45,7 +45,7 @@ All counters are `Counter<long>`. Tag values come from the call sites listed und
| `mxgateway.workers.killed` | `reason` | Forced terminations of worker processes. |
| `mxgateway.workers.exited` | `reason` | Clean or fault-driven worker exits. |
| `mxgateway.heartbeats.failed` | *(none)* | Aggregate worker heartbeat misses. The exported counter carries **no** `session_id` tag — per-session attribution is unbounded cardinality (every session mints a new time series), so it is kept out of the exporter; use the dashboard snapshot or the log scope for per-session detail. |
| `mxgateway.grpc.streams.disconnected` | `reason` | Detachments of the dashboard or client gRPC event stream. |
| `mxgateway.grpc.streams.disconnected` | `reason` | Ends of a client `StreamEvents` stream. `reason` is one of a fixed set (`EventStreamDisconnectReasons`): `SessionClosed` (the session event stream completed), `Overflow` (subscriber fell behind its bounded queue), `WorkerFault` (the worker event stream faulted), `ClientCancelled` (the client cancelled or disconnected), `Error` (unclassified), `Detached` (the consumer stopped reading without cancelling). |
| `mxgateway.retries.attempted` | `area` | Resilience retries executed by gateway components. |
### Histograms
@@ -181,7 +181,7 @@ IDisposable backlogRegistration = metrics.RegisterEventStreamBacklogSource(
() => subscriber.Reader.CanCount ? subscriber.Reader.Count : 0);
...
backlogRegistration.Dispose();
metrics.StreamDisconnected("Detached");
metrics.StreamDisconnected(disconnectReason); // an EventStreamDisconnectReasons constant
...
metrics.QueueOverflow("grpc-event-stream");
metrics.Fault(SessionManagerErrorCode.EventQueueOverflow.ToString());