fix(worker): review follow-up — ghost cache entries, ReadBulk dedupe/pumping, fault hygiene

- AddItem/AddItem2/AddBufferedItem drop stale cache entries for a reused handle id
- ReadBulk: dedupe handles within a batch (advise/teardown once, fan out result);
  pump every 16 items in add/advise and teardown phases
- alarm poll stops without recording a fault once StaRuntime.LoopFault is set
- StaRuntime skips the loop-fault handler during a requested shutdown
- StaHung loop fault diagnostic carries type + HRESULT only
- gateway.md documents ReadBulk's shared deadline

clients/rust/crates/mxgw-cli/src/main.rs

@@ -374,7 +374,9 @@ enum Command {
         reference: String,
         #[arg(long, default_value = "")]
         comment: String,
-        #[arg(long, default_value = "")]
+        /// Galaxy account of the acknowledging operator (required; the gateway
+        /// rejects an empty operator so the real operator is recorded).
+        #[arg(long)]
         operator: String,
         /// Galaxy domain of the operator; empty authenticates the bare user name.
         #[arg(long, default_value = "")]
@@ -2790,6 +2792,17 @@ mod tests {
         assert!(parsed.is_ok(), "parse failed: {parsed:?}");
     }
 
+    #[test]
+    fn acknowledge_alarm_requires_operator() {
+        let parsed = Cli::try_parse_from([
+            "mxgw",
+            "acknowledge-alarm",
+            "--reference",
+            "Tank01.Level.HiHi",
+        ]);
+        assert!(parsed.is_err());
+    }
+
     #[test]
     fn acknowledge_alarm_requires_reference() {
         let parsed = Cli::try_parse_from(["mxgw", "acknowledge-alarm"]);

docs/AlarmClientDiscovery.md

@@ -1051,6 +1051,18 @@ Every transition and snapshot record carries `degraded = true` and
 `source_provider = ALARM_PROVIDER_MODE_SUBTAG`. Clients that require full
 fidelity must wait for failback to the alarm manager.
 
+### Retrigger transitions
+
+`ALARM_TRANSITION_KIND_RETRIGGER` is reserved in the contract and the worker
+never emits it, in either provider mode. The alarmmgr XML reports only the
+four states `UNACK_ALM`, `ACK_ALM`, `UNACK_RTN`, and `ACK_RTN`, so a condition
+that returns to alarm after clearing (`*_RTN` → `*_ALM`) looks the same as a
+new raise, and `AlarmRecordTransitionMapper.MapTransition` reports it as
+`ALARM_TRANSITION_KIND_RAISE`. The subtag source has no re-alarm counter to
+observe either. The worker does not infer a retrigger from timing or
+from a raise that follows a clear, because the gateway forwards only
+transitions its source reports.
+
 ### Provider mode reflection
 
 When `FailoverAlarmConsumer` switches between providers, it raises

docs/AlarmProbeFindings.md

@@ -492,3 +492,56 @@ The gateway acknowledges by authenticating the operator and writing `AckMsg`
 (`docs/DesignDecisions.md`, "Alarm acknowledgement writes AckMsg as the
 authenticated operator"). The managed ack consumer and the second wnwrap ack
 consumer were removed.
+
+### Gateway-level verification
+
+The same result holds end to end through the gateway. With the feature build deployed on
+the windev rig and `TestAlarm001` raised, an `AcknowledgeAlarm` RPC for
+`Galaxy!TestArea.TestMachine_001.TestAlarm001` with `operator_user=Administrator` and an
+empty password returned `PROTOCOL_STATUS_CODE_OK`, `hresult` 0. `.Acked` turned True,
+`.AckMsg` held the comment, and the Historian logged the `ACK_ALM` row with
+`Operator=Administrator`, `DomainName=ArchestrA`, `UserFullName=Administrator`,
+`OperatorNode=MxGateway` (the default `AckClientName`), and `Description` = the comment.
+
+A wrong password, or a user Galaxy security does not know, makes `AuthenticateUser`
+throw `ArgumentException` (`E_INVALIDARG`, 0x80070057) rather than a `COMException`.
+That is why the writer treats `ArgumentException` from the MXAccess call itself as an
+interop failure alongside `COMException`, and reports the HRESULT the exception carries.
+Such an ack returns `PROTOCOL_STATUS_CODE_MXACCESS_FAILURE`, the alarm stays unacked,
+the Historian gains no ack row, and the password string does not appear in the gateway
+log, `stdout.log`, or `stderr.log`. A non-Galaxy provider reference returns
+`PROTOCOL_STATUS_CODE_INVALID_REQUEST` with `hresult` -1, and an empty `operator_user`
+returns gRPC `InvalidArgument`.
+
+The GUID arm of the ack command was not driven live. The public alarm RPCs expose only
+the full reference, so clients reach the by-name arm; worker unit tests cover the GUID
+arm.
+
+## Item handle reuse
+
+`AlarmAckWriter` correlates an ack with its `OnWriteComplete` by the
+(server handle, item handle) pair, because the callback carries no per-write id.
+After an unconfirmed ack it drops the `AckMsg` item and re-adds it so the next
+ack waits on a new handle that a late completion for the old one cannot match.
+That only works if MXAccess does not hand the same number out again.
+
+`ItemHandleReuseProbeTests` (Skip-gated, worker tests) checked this on windev
+(`10.100.0.48`, Galaxy `DEV`) on 2026-10-20 against
+`TestMachine_001.TestAlarm001.AckMsg`, without writing:
+
+| Sequence on one registration (server handle 1) | Item handles returned |
+|---|---|
+| `AddItem` + `Advise`, then `UnAdvise` + `RemoveItem`, five times | 1, 2, 3, 4, 5 |
+| `AddItem`, then `RemoveItem`, five times | 6, 7, 8, 9, 10 |
+| Add and remove `AckMsg`, add `Acked`, add `AckMsg` again | 11, then 12 (`Acked`), 13 |
+| Add `AckMsg` twice while both are held | 14, 15 |
+
+Within a registration the numbers increase and a freed number is never reused,
+so the drop-and-re-add mitigation holds. Numbering is per registration:
+`Unregister` + `Register` on the same COM object returned server handle 2 and
+item handle 1, and a new `LMXProxyServerClass` returned server handle 1 and
+item handle 1 again. The writer is not exposed to that restart because its
+teardown detaches its `OnWriteComplete` handler from the old COM object and
+clears that server handle's completion entries before it creates a new one.
+No per-handle generation counter is needed.
+

docs/Authorization.md

@@ -125,7 +125,7 @@ public string ResolveRequiredScope(object request)
 }
 ```
 
-The `_ => GatewayScopes.Admin` fallback is intentional: any future request type that the resolver does not recognize fails closed, requiring the strongest scope until the resolver is updated. `AcknowledgeAlarm` is treated as a write — it mutates alarm state, mirroring `MxCommandKind.Write*`. It stays `invoke:write` rather than `invoke:secure` because the request carries the operator's own Galaxy credential, which the worker authenticates before the `AckMsg` write; the credential, not the API-key scope, authorizes the identity the Historian records. `StreamAlarms` and `QueryActiveAlarms` share the alarm/event surface with `StreamEvents` and `MxCommandKind.DrainEvents`, so they carry `events:read` (the active-alarm snapshot is the same data reachable through the event surface). All three alarm RPCs are session-less: the scope check is the only authorization gate, since there is no per-session ownership to enforce.
+The `_ => GatewayScopes.Admin` fallback is intentional: any future request type that the resolver does not recognize fails closed, requiring the strongest scope until the resolver is updated. `AcknowledgeAlarm` is treated as a write — it mutates alarm state, mirroring `MxCommandKind.Write*`. It stays `invoke:write` rather than `invoke:secure` because the request carries the operator's own Galaxy credential, which the worker authenticates before the `AckMsg` write; the credential, not the API-key scope, authorizes the identity the Historian records. The trade-off is that an `invoke:write` key can tell `AuthenticationFailed` (an `MXACCESS_FAILURE` whose diagnostic reports operator authentication failure) apart from every other outcome, which makes `AcknowledgeAlarm` a Galaxy-password oracle for that key — whereas the `AuthenticateUser` command itself requires `invoke:secure`. What bounds it: the caller must already hold a valid, admin-issued, revocable `invoke:write` key, and every acknowledgement is serialized through the single monitor worker session's STA, so guesses run one at a time. The gateway does **not** throttle failed operator authentications: the `ApiKeyFailureLimiter` counts only failed API-key verifications, and the ASP.NET Core rate limiter covers only the dashboard `/login` endpoint. Any lockout after repeated bad passwords comes from the Galaxy security configuration, not from the gateway. Grant `invoke:write` only to clients trusted with that exposure. `StreamAlarms` and `QueryActiveAlarms` share the alarm/event surface with `StreamEvents` and `MxCommandKind.DrainEvents`, so they carry `events:read` (the active-alarm snapshot is the same data reachable through the event surface). All three alarm RPCs are session-less: the scope check is the only authorization gate, since there is no per-session ownership to enforce.
 
 `MxCommandRequest` is special because it multiplexes many MxAccess operations through a single RPC. The resolver inspects the embedded `MxCommandKind` so each operation gets its own scope:
 

docs/DesignDecisions.md

@@ -818,6 +818,14 @@ per-request credential is workable. See
 - The managed ack consumer, the WNAL native ack consumer, the second wnwrap
   ack-only consumer and the opt-in managed-ack environment flag are removed.
 
+**Upgrade note.** This is a breaking change for callers. They must now send
+`operator_user` (required; empty is rejected with `InvalidArgument`) and the
+operator's Galaxy password; a caller that sends no password for a
+password-protected account gets `MXACCESS_FAILURE` from `AuthenticateUser`
+instead of an acknowledgement. OtOpcUa's `GalaxyDriver` alarm acknowledger is a
+known caller that must be updated to forward the operator's credentials before
+the gateway is upgraded.
+
 ## Related Documentation
 
 - [Gateway Process Detailed Design](./GatewayProcessDesign.md)

docs/Grpc.md

@@ -110,11 +110,17 @@ The worker calls `AuthenticateUser` as the operator, then writes `operator_comme
 |---|---|---|
 | Write applied, or sent but no `OnWriteComplete` within the wait (unconfirmed, with a diagnostic) | `OK` | `0` |
 | Unknown alarm, or no ack target (non-Galaxy provider, no ack-comment subtag) | `INVALID_REQUEST` | `-1` |
-| `AuthenticateUser` failed | `MXACCESS_FAILURE` | the `AuthenticateUser` HRESULT |
-| Write rejected (`OnWriteComplete` non-OK) | `MXACCESS_FAILURE` | the `detail` of the `OnWriteComplete` `MXSTATUS_PROXY` |
+| `AuthenticateUser` raised an MXAccess error (bad user or wrong password surfaces as `E_INVALIDARG` `0x80070057`, raised by the interop as `ArgumentException`; `COMException` and `UnauthorizedAccessException` are classified the same way) | `MXACCESS_FAILURE` | the exception's HRESULT |
+| `AuthenticateUser` returned user id `0` | `MXACCESS_FAILURE` | `-1` |
+| `AuthenticateUser` failed with a server-gone HRESULT (`RPC_E_DISCONNECTED`, `RPC_S_SERVER_UNAVAILABLE`, `RPC_E_SERVERFAULT`); the writer tears down its MXAccess registration and re-registers on the next ack | `MXACCESS_FAILURE` | that HRESULT |
+| `AddItem`, `Advise` or `Write` on the `AckMsg` item raised an MXAccess error (`COMException`, or `ArgumentException` for `E_INVALIDARG`); the item is dropped and re-added on the next ack, and a server-gone HRESULT also tears down the registration | `MXACCESS_FAILURE` | the exception's HRESULT |
+| Write rejected (`OnWriteComplete` non-OK) | `MXACCESS_FAILURE` | the `detail` of the `OnWriteComplete` `MXSTATUS_PROXY`, or `-1` when `detail` is `0` |
+| Any other exception escaping the ack (e.g. the ack writer's MXAccess `Register` failing, the monitor's worker session having no active alarm subscription — `InvalidOperationException` — or an exception from the worker's own code rather than from an MXAccess call: interop failures are classified only when the `AuthenticateUser`, `AddItem`, `Advise` or `Write` call itself raises them, and an `ArgumentNullException` or `ArgumentOutOfRangeException` is never classified because the interop does not raise them for an HRESULT) | `MXACCESS_FAILURE` | the exception's `HResult`; the diagnostic names only the exception type and HRESULT |
 
 `reply.Statuses` carries the `OnWriteComplete` `MxStatusProxy` rows when they were observed. Because an unconfirmed write also returns `OK`, a client that must be certain should watch for the resulting transition on `StreamAlarms`. `AcknowledgeAlarm` stays under the `invoke:write` scope; see [Authorization](./Authorization.md).
 
+**Upgrade note.** Callers must now send `operator_user` (required) and the operator's Galaxy password in `operator_password`. A caller that sends no `operator_user` gets `InvalidArgument`; a caller that sends no password for a password-protected Galaxy account gets `MXACCESS_FAILURE` from `AuthenticateUser` instead of an acknowledgement. Every existing caller must be updated before the gateway is upgraded — known callers include OtOpcUa's `GalaxyDriver` alarm acknowledger, which must supply the operator's credentials.
+
 ### `StreamAlarms`
 
 `StreamAlarms` is a server-streaming, **session-less** RPC that attaches to the gateway's central alarm feed. The handler delegates to `IGatewayAlarmService.StreamAsync`. The stream opens with a `provider_status` and a `snapshot_status` `AlarmFeedMessage` (the current provider mode and snapshot-completeness verdict), then one `AlarmFeedMessage` carrying an `active_alarm` per currently-active alarm (the ConditionRefresh snapshot), then a single `snapshot_complete`, then a `transition` for every subsequent raise / acknowledge / clear — interleaved with a further `provider_status` on each failover/failback and a further `snapshot_status` on each change of the truncation verdict. It is served by the always-on `GatewayAlarmMonitor`, which owns a single gateway-managed worker session and fans out to every attached client — clients no longer open a session of their own. `alarm_filter_prefix`, when set, scopes the stream to a sub-tree.

docs/MxAccessWorkerInstanceDesign.md

@@ -133,6 +133,9 @@ starts the STA, or touches MXAccess:
 | `2` | `InvalidArguments` | Required arguments are missing or unknown arguments are present. |
 | `3` | `InvalidProtocolVersion` | `--protocol-version` is not numeric or does not match the supported worker protocol. |
 | `4` | `MissingNonce` | `MXGATEWAY_WORKER_NONCE` is absent or empty. |
+| `5` | `PipeConnectionFailed` | An `IOException` or `TimeoutException` escapes the pipe client. |
+| `6` | `ProtocolViolation` | A `WorkerFrameProtocolException` escapes the pipe client. |
+| `7` | `UnhandledException` | An exception reaches the `AppDomain.UnhandledException` hook (see `docs/WorkerBootstrap.md`). |
 
 Bootstrap logs use `WorkerConsoleLogger` key/value output. `WorkerLogRedactor`
 redacts fields whose names indicate nonce, secret, password, token,
@@ -270,12 +273,33 @@ is still responsive. Shutdown marks the runtime as closing, wakes the pump,
 rejects new commands, cancels queued work, uninitializes COM on the STA, and
 waits for the thread to exit.
 
+`ProcessQueuedCommands` pumps pending Windows messages between queued work
+items whenever more work remains, not only after the whole queue drains. A
+burst of queued commands would otherwise hold every inbound COM callback
+(`OnDataChange`, `OnWriteComplete`) behind the entire backlog.
+
+If the STA loop itself throws after `Start` has returned (a failed
+`MsgWaitForMultipleObjectsEx`, or an exception escaping a dispatched message),
+`StaRuntime` records the exception as `LoopFault`, fails every later
+`InvokeAsync` with an `InvalidOperationException` wrapping it (never the
+graceful `StaRuntimeShutdownException`), cancels queued work, and invokes the
+handler registered through `RegisterLoopFaultHandler` — unless a requested
+shutdown is already in progress, in which case the death is part of a
+deliberate teardown and is not reported. `MxAccessStaSession` registers a
+handler that records a `StaHung` `WorkerFault` on the event queue (diagnostic
+text carries the exception type and HRESULT only, never the message),
+so the IPC drain loop sends the fault to the gateway and ends the pipe session
+and the worker process. Before this, a post-start loop death was stored in the
+startup-exception field that only `Start` reads, leaving a live worker whose
+commands queued forever. Startup failures still surface from `Start` itself.
+
 ### Inner Completion Waits
 
 Two commands hold the STA while waiting for a COM event they just provoked: the
 unary write path waits for its `OnWriteComplete`
-(`MxAccessWriteCompletionCache.TryWaitForCompletion`), and `ReadBulk` waits per
-tag for the first `OnDataChange` (`MxAccessValueCache.TryWaitForUpdate`). Both
+(`MxAccessWriteCompletionCache.TryWaitForCompletion`), and `ReadBulk` waits for
+the first `OnDataChange` of every snapshot tag
+(`MxAccessValueCache.TryWaitForUpdates`). Both
 run the same loop shape as the outer pump, for the same reason — the event they
 are waiting for *is* a Windows message, so the thread must keep dispatching to
 receive it:
@@ -297,7 +321,7 @@ The idle slice is a Win32 wait (`StaWaitHelper.WaitForSignalOrMessages`), never
 `Thread.Sleep`. A sleeping STA pumps no messages, so a sleep-polled loop could
 only dispatch the awaited COM event at poll-tick granularity while stalling
 *every other* event for the same tick — up to 1.5 s for a write completion and
-up to `timeout_ms` per tag for `ReadBulk`. The Win32 wait returns the instant a
+up to `timeout_ms` for `ReadBulk`. The Win32 wait returns the instant a
 message needs pumping, so the apartment dispatches continuously for the whole
 wait. Each cache also sets an `AutoResetEvent` from its update path (outside the
 cache lock) so a cross-thread producer wakes the waiter immediately; in the live
@@ -317,8 +341,29 @@ nothing wakes the wait — a process with no STA message queue (unit tests drive
 these caches from ordinary threads, standing in for the STA by updating the
 cache from a fake `pumpStep`) must not block for a full poll interval. Timeouts,
 deadline math, and return values are unchanged by the wait mechanism: an expired
-write wait still yields the empty-`statuses` unconfirmed reply, and an expired
-per-tag `ReadBulk` wait still reports its own timeout.
+write wait still yields the empty-`statuses` unconfirmed reply, and every
+`ReadBulk` tag that did not deliver before the deadline still reports its own
+timeout.
+
+`ReadBulk` runs its snapshot lifecycle in phases so the STA is held for at most
+one `timeout_ms` per command, not one per tag: (1) every tag is validated and
+probed against the cached-and-advised fast path; (2) every remaining tag is
+added and advised; (3) one wait with a shared deadline collects the first
+`OnDataChange` per snapshot handle and returns as soon as all have delivered;
+(4) every snapshot handle is unadvised and removed, best-effort. Per-tag
+results are unchanged: request order, the blank-address failure, the cached
+result, an `AddItem` failure (item handle 0) or `Advise` failure (its own item
+handle) failing only that tag, and the `ReadBulk timed out after <n> ms waiting
+for first OnDataChange.` message for each tag that never delivered. The cache
+probes all run before any snapshot `AddItem`, so a later tag in the same
+request can never be served from an earlier tag's snapshot subscription.
+When MXAccess returns the same item handle for two tags in one batch (a
+duplicate address), the handle is advised, unadvised, and removed once, and
+every duplicate receives the shared outcome. Phases 2 and 4 make one or two
+COM calls per tag without waiting, so they call the pump step every
+`ReadBulkPumpInterval` (16) tags; a large batch therefore keeps dispatching
+inbound callbacks and refreshing `LastStaActivityUtc` instead of drifting
+toward `HeartbeatStuckCeiling`.
 
 The write wait's budget is `MxGateway:Worker:WriteCompletionWaitMilliseconds`
 (default 1500). It applies to `Write`, `Write2`, `WriteSecured`, and
@@ -351,6 +396,23 @@ native client correlating by handle has the identical ambiguity). The exposure
 is bounded: `statuses` is an enrichment field, and `Hresult`/`ProtocolStatus`
 stay acceptance-only regardless of which completion (if any) was attributed.
 
+**The alarm ack writer narrows the window for acknowledgements.**
+`AlarmAckWriter` registers its own MXAccess server with its own
+`MxAccessWriteCompletionCache` and reuses one advised `AckMsg` item per alarm
+(least-recently-used, capped at 256). After an ack ends unconfirmed it marks the
+item dirty, and the next ack to that alarm removes and re-adds the item so the
+write waits on a fresh item handle the stale completion cannot match. This
+relies on MXAccess never reusing an item handle number within one
+registration, which a live probe confirmed (see "Item handle reuse" in
+[Alarm Probe Findings](./AlarmProbeFindings.md)). Every
+drop of an item (dirty re-add, eviction, or a failed `AddItem`/`Advise`/`Write`)
+also removes that handle's completion entry, and tearing down the registration
+removes every entry for its server handle, so the cache does not grow by one
+entry per dropped handle for the life of the worker. The writer records a
+completion only for an item handle it currently holds on its live
+registration, so a late `OnWriteComplete` for a dropped handle cannot recreate
+its entry.
+
 ## Live alarm watch-list replacement
 
 An auto-mode `SubscribeAlarms` with `fallback_enabled = true` always builds a
@@ -654,7 +716,20 @@ The worker records plain and supervisory advice separately only after the COM
 call returns normally. Successful `UnAdvise` removes all tracked advice for the
 server and item pair because the public MXAccess cleanup method has no plain
 versus supervisory selector. Successful `RemoveItem` and `Unregister` also clear
-related advice state from the worker registry. Failed advice and cleanup calls
+related advice state from the worker registry, and evict the per-handle
+caches the event sink fills: `RemoveItem` drops the handle pair's
+`MxAccessValueCache` and `MxAccessWriteCompletionCache` entries, and
+`Unregister` drops every entry recorded under the server handle (MXAccess
+removes the server's items with it, so the per-item eviction never runs for
+them). Without this, an `AddItem` → `Write` → `RemoveItem` churn grew the
+completion cache by one entry per cycle for the life of the worker.
+`AddItem`, `AddItem2`, and `AddBufferedItem` also drop both cache entries for
+the handle they receive when the registry does not already track it. An
+`OnDataChange` or `OnWriteComplete` that was queued when `RemoveItem` or
+`Unregister` ran is dispatched after the eviction and recreates the entry;
+when MXAccess later reuses that item id, `ReadBulk`'s cached path would
+otherwise serve the stale value as `was_cached = true`. A handle the
+registry still tracks belongs to a live item, so its entries are kept. Failed advice and cleanup calls
 leave registry state unchanged so diagnostics continue to reflect the last
 successful MXAccess-owned state transition.
 
@@ -888,6 +963,23 @@ the STA more than 75 s without pumping *should* fault — that is the ceiling's
 contract). The flag is named generically so any future non-dispatcher STA work
 reuses it.
 
+A `PollOnce` that throws does not fault the session on the first failure. The
+poll loop counts consecutive failures and records the alarm-poll fault
+(`MxaccessEventConversionFailed`) only when the count reaches
+`MxAccessStaSession.DefaultAlarmPollFailureThreshold` (3, the same default as
+`AlarmFailoverConfig.consecutive_failure_threshold`); a successful poll resets
+the count. Each failure below the threshold writes one
+`level=Warning event=AlarmPollFailed` line to stderr carrying the exception
+type, HRESULT, and counts only — the exception message is left out because it
+can echo alarm or tag names. A single transient COM failure in
+`GetXmlCurrentAlarms2` used to fault the whole session, taking every data
+subscription with it; a persistent failure, including an STA-affinity
+regression that fails every tick, still faults within three poll intervals.
+A poll failure seen after the STA loop itself has died (`StaRuntime.LoopFault`
+is set) ends the poll loop without recording anything: the loop-fault handler
+has already recorded the `StaHung` fault, and the failure is only the
+fast-fail `InvokeAsync` returns afterwards.
+
 The in-flight-command suppression itself is bounded by
 `WorkerPipeSessionOptions.HeartbeatStuckCeiling` (default 75 seconds = 5 ×
 `HeartbeatGrace`). The motivating case for the suppression is a legitimately

docs/WorkerBootstrap.md

@@ -117,8 +117,13 @@ public static WorkerBootstrapResult Failure(WorkerExitCode exitCode, IEnumerable
 | `MissingNonce` | 4 | The `MXGATEWAY_WORKER_NONCE` environment variable is null, empty, or whitespace. |
 | `PipeConnectionFailed` | 5 | An `IOException` or `TimeoutException` escapes the pipe client. |
 | `ProtocolViolation` | 6 | A `WorkerFrameProtocolException` escapes the pipe client. |
+| `UnhandledException` | 7 | An exception reaches `AppDomain.CurrentDomain.UnhandledException` — thrown on a thread outside `WorkerApplication.Run`'s `try/catch` (a background task continuation, a timer, a COM callback thread). |
 
-`InvalidArguments`, `InvalidProtocolVersion`, and `MissingNonce` originate in the parser; the others originate in `WorkerApplication.Run`'s `try/catch` around the pipe client.
+`InvalidArguments`, `InvalidProtocolVersion`, and `MissingNonce` originate in the parser; `UnhandledException` originates in the process-wide hook described below; the others originate in `WorkerApplication.Run`'s `try/catch` around the pipe client.
+
+### Unhandled-exception hook
+
+The production entry point, `WorkerApplication.Run(string[])`, subscribes `AppDomain.CurrentDomain.UnhandledException` before it parses arguments. The handler (`WorkerApplication.HandleUnhandledException`) logs one `WorkerUnhandledException` error line and calls `Environment.Exit(7)`. Without it, an exception on a thread that `Run` does not own ended the process with the CLR's crash code and no worker log line, so the gateway saw an unexplained exit. The line carries `exit_code`, `exception_type`, and `is_terminating` only: the exception message and stack are left out because they can carry tag values or credentials passed through a command, the same rule every other worker failure line follows. A failure inside the logger is swallowed so the exit always happens. The injectable `Run` overloads that tests call do not install the hook, because it is process-global.
 
 ## Environment Abstraction
 

docs/WorkerSta.md

@@ -86,10 +86,16 @@ StaWorkItem<T> workItem = new(command, cancellationToken);
 
 lock (gate)
 {
+    if (loopFault is not null)
+    {
+        return Task.FromException<T>(new InvalidOperationException(
+            "The worker STA loop terminated unexpectedly.",
+            loopFault));
+    }
+
     if (shutdownRequested)
     {
-        return Task.FromException<T>(
-            new InvalidOperationException("The worker STA runtime is shutting down."));
+        return Task.FromException<T>(new StaRuntimeShutdownException());
     }
 
     commandQueue.Enqueue(workItem);
@@ -101,7 +107,11 @@ return workItem.Task;
 
 `StaWorkItem<T>` uses an `Interlocked.CompareExchange` on `started` so that exactly one of three outcomes happens: the STA thread executes the delegate, an external `CancellationToken` cancellation fires first, or `CancelBeforeExecution` runs during shutdown. `Execute` runs on the STA thread, sets `Completion` from `command()`, and propagates exceptions through `TrySetException` so the awaiting caller observes them.
 
-`ProcessQueuedCommands` is the only consumer of the queue and runs on the STA thread, so each work item is guaranteed to execute on the apartment that owns the COM object.
+`ProcessQueuedCommands` is the only consumer of the queue and runs on the STA thread, so each work item is guaranteed to execute on the apartment that owns the COM object. It calls `PumpPendingMessages` between work items whenever more work is queued, so a backlog of commands cannot hold inbound COM callbacks until the whole queue drains.
+
+### Loop failure after start
+
+An exception thrown by the loop after `Start()` returned — `WaitForWorkOrMessages` reporting `WAIT_FAILED`, or an exception escaping a dispatched message — is not a startup failure, and nothing reads `startupException` after `Start()`. `ThreadMain` instead stores it as `LoopFault` under `gate`, so every later `InvokeAsync` fails immediately with an `InvalidOperationException` whose inner exception is the loop fault, and work already queued is cancelled by the `finally` block. It then invokes the handler registered with `RegisterLoopFaultHandler`; a handler registered after the loop died runs immediately on the registering thread. Neither happens when `Shutdown` was already requested, because a loop that throws during a deliberate teardown is not a worker fault. `MxAccessStaSession` registers a handler that records a `StaHung` `WorkerFault` on the event queue, which the IPC drain loop sends to the gateway before ending the pipe session and the worker process.
 
 ## Command Dispatch
 

gateway.md

@@ -569,8 +569,15 @@ returns one result per requested entry — per-entry failures populate
   advised (`was_cached = true`, no subscription side-effects), or by
   taking a full `AddItem` + `Advise` + wait-for-first-OnDataChange +
   `UnAdvise` + `RemoveItem` snapshot lifecycle when no live subscription
-  exists (`was_cached = false`). Per-tag timeouts surface as
-  `was_successful = false` rather than throwing. The cache lives on the
+  exists (`was_cached = false`). `timeout_ms` is one shared deadline for
+  the whole command, not a per-tag budget: the worker adds and advises
+  every uncached tag first, waits once for all of them, then tears them
+  all down, so the STA is held for at most one `timeout_ms` (plus the
+  add/advise and teardown COM calls, during which it pumps every 16 tags).
+  Each tag that has not delivered by the deadline surfaces its own
+  `was_successful = false` timeout rather than throwing. Duplicate
+  addresses that MXAccess resolves to the same item handle share one
+  subscription and receive the same result. The cache lives on the
   worker's `MxAccessValueCache`, populated by `MxAccessBaseEventSink` on
   every `OnDataChange` after the event clears the outbound queue.
   Required scope: `invoke:read`. `timeout_ms == 0` uses the worker's

src/ZB.MOM.WW.MxGateway.IntegrationTests/WorkerLiveMxAccessSmokeTests.cs

@@ -1052,6 +1052,231 @@ public sealed class WorkerLiveMxAccessSmokeTests(ITestOutputHelper output)
         }
     }
 
+    /// <summary>
+    /// TST-05: the real worker answers the four non-terminating control commands off-STA in
+    /// <c>WorkerPipeSession.HandleControlCommandAsync</c>. The default suite only ever sees the
+    /// FakeWorkerHarness canned replies, so this drives <c>Ping</c>, <c>GetSessionState</c>,
+    /// <c>GetWorkerInfo</c>, and <c>DrainEvents</c> through the gateway against a live worker and
+    /// asserts each reply carries real worker state rather than an <c>INVALID_REQUEST</c>.
+    /// </summary>
+    /// <returns>A task that represents the asynchronous operation.</returns>
+    [LiveMxAccessFact]
+    public async Task GatewaySession_WithLiveWorker_ControlCommands_ReturnRealWorkerState()
+    {
+        string workerExecutablePath = IntegrationTestEnvironment.ResolveLiveMxAccessWorkerExecutablePath();
+        Assert.True(
+            File.Exists(workerExecutablePath),
+            $"Live MXAccess worker executable was not found at {workerExecutablePath}. Build the worker or set {IntegrationTestEnvironment.LiveMxAccessWorkerExecutableVariableName}.");
+
+        TestWorkerProcessFactory processFactory = new(output);
+        await using GatewayServiceFixture fixture = new(workerExecutablePath, processFactory, output);
+
+        string? sessionId = null;
+
+        try
+        {
+            OpenSessionReply openReply = await fixture.Service.OpenSession(
+                new OpenSessionRequest
+                {
+                    ClientSessionName = "live-mxaccess-control",
+                    ClientCorrelationId = "live-open-control",
+                    CommandTimeout = Duration.FromTimeSpan(CommandTimeout),
+                },
+                new TestServerCallContext()).ConfigureAwait(false);
+
+            sessionId = openReply.SessionId;
+            Assert.Equal(ProtocolStatusCode.Ok, openReply.ProtocolStatus.Code);
+            Assert.True(openReply.WorkerProcessId > 0);
+
+            MxCommandReply pingReply = await fixture.Service.Invoke(
+                CreateControlRequest(sessionId, "live-ping", new MxCommand
+                {
+                    Kind = MxCommandKind.Ping,
+                    Ping = new PingCommand { Message = "live-ping-echo" },
+                }),
+                new TestServerCallContext()).ConfigureAwait(false);
+            LogReply("Ping", pingReply);
+            Assert.Equal(ProtocolStatusCode.Ok, pingReply.ProtocolStatus.Code);
+            Assert.Equal(MxCommandKind.Ping, pingReply.Kind);
+
+            MxCommandReply stateReply = await fixture.Service.Invoke(
+                CreateControlRequest(sessionId, "live-get-session-state", new MxCommand
+                {
+                    Kind = MxCommandKind.GetSessionState,
+                    GetSessionState = new GetSessionStateCommand(),
+                }),
+                new TestServerCallContext()).ConfigureAwait(false);
+            LogReply("GetSessionState", stateReply);
+            Assert.Equal(ProtocolStatusCode.Ok, stateReply.ProtocolStatus.Code);
+            Assert.Equal(MxCommandKind.GetSessionState, stateReply.Kind);
+            Assert.NotNull(stateReply.SessionState);
+            Assert.Equal(SessionState.Ready, stateReply.SessionState.State);
+
+            MxCommandReply infoReply = await fixture.Service.Invoke(
+                CreateControlRequest(sessionId, "live-get-worker-info", new MxCommand
+                {
+                    Kind = MxCommandKind.GetWorkerInfo,
+                    GetWorkerInfo = new GetWorkerInfoCommand(),
+                }),
+                new TestServerCallContext()).ConfigureAwait(false);
+            LogReply("GetWorkerInfo", infoReply);
+            Assert.Equal(ProtocolStatusCode.Ok, infoReply.ProtocolStatus.Code);
+            Assert.Equal(MxCommandKind.GetWorkerInfo, infoReply.Kind);
+            Assert.NotNull(infoReply.WorkerInfo);
+            output.WriteLine(
+                $"GetWorkerInfo pid={infoReply.WorkerInfo.WorkerProcessId} version={infoReply.WorkerInfo.WorkerVersion} "
+                + $"progid={infoReply.WorkerInfo.MxaccessProgid} clsid={infoReply.WorkerInfo.MxaccessClsid}");
+            Assert.Equal(openReply.WorkerProcessId, infoReply.WorkerInfo.WorkerProcessId);
+            Assert.False(string.IsNullOrWhiteSpace(infoReply.WorkerInfo.WorkerVersion));
+            Assert.False(string.IsNullOrWhiteSpace(infoReply.WorkerInfo.MxaccessProgid));
+            Assert.False(string.IsNullOrWhiteSpace(infoReply.WorkerInfo.MxaccessClsid));
+
+            // No StreamEvents subscriber is attached, so whatever the worker queued since start
+            // is still drainable; the reply must carry the DrainEvents payload (possibly empty).
+            MxCommandReply drainReply = await fixture.Service.Invoke(
+                CreateControlRequest(sessionId, "live-drain-events", new MxCommand
+                {
+                    Kind = MxCommandKind.DrainEvents,
+                    DrainEvents = new DrainEventsCommand { MaxEvents = 16 },
+                }),
+                new TestServerCallContext()).ConfigureAwait(false);
+            LogReply("DrainEvents", drainReply);
+            Assert.Equal(ProtocolStatusCode.Ok, drainReply.ProtocolStatus.Code);
+            Assert.Equal(MxCommandKind.DrainEvents, drainReply.Kind);
+            Assert.Equal(MxCommandReply.PayloadOneofCase.DrainEvents, drainReply.PayloadCase);
+            Assert.True(drainReply.DrainEvents.Events.Count <= 16);
+            output.WriteLine($"DrainEvents drained={drainReply.DrainEvents.Events.Count}");
+
+            // A control command must not disturb the session: it is still Ready afterwards.
+            Assert.True(fixture.TryGetSession(sessionId, out GatewaySession? session));
+            Assert.Equal(SessionState.Ready, session!.State);
+        }
+        finally
+        {
+            await ShutDownAsync(fixture, processFactory, sessionId, streamTask: null).ConfigureAwait(false);
+        }
+    }
+
+    /// <summary>
+    /// TST-05: <c>ShutdownWorker</c> writes its OK reply before the worker shuts down, so the
+    /// reply reaches the caller (a reply written after the pipe closed would surface as a
+    /// pipe-disconnected failure instead), the worker process then exits on its own, and a later
+    /// command on the session does not succeed. Uses its own fixture and worker because it
+    /// terminates the worker.
+    /// </summary>
+    /// <returns>A task that represents the asynchronous operation.</returns>
+    [LiveMxAccessFact]
+    public async Task GatewaySession_WithLiveWorker_ShutdownWorker_RepliesOkThenWorkerExits()
+    {
+        string workerExecutablePath = IntegrationTestEnvironment.ResolveLiveMxAccessWorkerExecutablePath();
+        Assert.True(
+            File.Exists(workerExecutablePath),
+            $"Live MXAccess worker executable was not found at {workerExecutablePath}. Build the worker or set {IntegrationTestEnvironment.LiveMxAccessWorkerExecutableVariableName}.");
+
+        TestWorkerProcessFactory processFactory = new(output);
+        await using GatewayServiceFixture fixture = new(workerExecutablePath, processFactory, output);
+        using RecordingServerStreamWriter<MxEvent> eventWriter = new();
+
+        string? sessionId = null;
+        Task? streamTask = null;
+        using CancellationTokenSource streamCancellation = new();
+
+        try
+        {
+            OpenSessionReply openReply = await fixture.Service.OpenSession(
+                new OpenSessionRequest
+                {
+                    ClientSessionName = "live-mxaccess-shutdown-worker",
+                    ClientCorrelationId = "live-open-shutdown-worker",
+                    CommandTimeout = Duration.FromTimeSpan(CommandTimeout),
+                },
+                new TestServerCallContext()).ConfigureAwait(false);
+
+            sessionId = openReply.SessionId;
+            Assert.Equal(ProtocolStatusCode.Ok, openReply.ProtocolStatus.Code);
+
+            // The gateway observes a worker's pipe closing through the event-stream read loop
+            // (the same path the abnormal-exit fact relies on); without a subscriber the session
+            // would sit in Ready until the heartbeat grace expired.
+            streamTask = fixture.Service.StreamEvents(
+                new StreamEventsRequest { SessionId = sessionId },
+                eventWriter,
+                new TestServerCallContext(streamCancellation.Token));
+
+            MxCommandReply shutdownReply = await fixture.Service.Invoke(
+                CreateControlRequest(sessionId, "live-shutdown-worker", new MxCommand
+                {
+                    Kind = MxCommandKind.ShutdownWorker,
+                    ShutdownWorker = new ShutdownWorkerCommand
+                    {
+                        GracePeriod = Duration.FromTimeSpan(TimeSpan.FromSeconds(5)),
+                    },
+                }),
+                new TestServerCallContext()).ConfigureAwait(false);
+            LogReply("ShutdownWorker", shutdownReply);
+            Assert.Equal(ProtocolStatusCode.Ok, shutdownReply.ProtocolStatus.Code);
+            Assert.Equal(MxCommandKind.ShutdownWorker, shutdownReply.Kind);
+
+            // The worker exits on its own — no CloseSession, no kill. WaitForProcessesAsync
+            // throws on timeout, which fails the test.
+            await processFactory.WaitForProcessesAsync(StreamShutdownTimeout).ConfigureAwait(false);
+
+            // The session can no longer serve commands once its worker is gone: either the
+            // gateway rejects the call outright or the reply is not Ok.
+            MxCommandReply? afterReply = null;
+            Exception? afterException = null;
+            try
+            {
+                afterReply = await fixture.Service.Invoke(
+                    CreateControlRequest(sessionId, "live-ping-after-shutdown", new MxCommand
+                    {
+                        Kind = MxCommandKind.Ping,
+                        Ping = new PingCommand(),
+                    }),
+                    new TestServerCallContext()).ConfigureAwait(false);
+                LogReply("PingAfterShutdown", afterReply);
+            }
+            catch (Exception exception)
+            {
+                afterException = exception;
+                output.WriteLine($"PingAfterShutdown rejected: {exception.GetType().Name}: {exception.Message}");
+            }
+
+            Assert.True(
+                afterException is not null || afterReply!.ProtocolStatus.Code != ProtocolStatusCode.Ok,
+                "A command sent after ShutdownWorker must not succeed against the exited worker.");
+
+            // Logged, not asserted: the gateway's WorkerClient moves to Closed on the shutdown
+            // ack, but GatewaySession.State is not driven from that transition today, so the
+            // public session state still reads Ready here (observed live on windev). That is a
+            // gateway-side gap, not a worker one — this fact pins the worker half of the
+            // contract: OK reply first, then a clean exit.
+            string observedState = fixture.TryGetSession(sessionId, out GatewaySession? session)
+                ? session.State.ToString()
+                : "<removed>";
+            output.WriteLine($"ShutdownWorker observed_session_state={observedState}");
+        }
+        finally
+        {
+            // The worker is gone; a CloseSession round-trip would only log a cleanup failure.
+            streamCancellation.Cancel();
+            await ShutDownAsync(fixture, processFactory, sessionId: null, streamTask).ConfigureAwait(false);
+        }
+    }
+
+    private static MxCommandRequest CreateControlRequest(
+        string sessionId,
+        string correlationId,
+        MxCommand command)
+    {
+        return new MxCommandRequest
+        {
+            SessionId = sessionId,
+            ClientCorrelationId = correlationId,
+            Command = command,
+        };
+    }
+
     /// <summary>
     /// Closes the session and drains the event stream / worker processes without letting a
     /// cleanup timeout mask the original failure from the test body.

src/ZB.MOM.WW.MxGateway.Server/ZB.MOM.WW.MxGateway.Server.csproj

@@ -32,6 +32,9 @@
          native 2.1.11; 2.1.12 patches it. Bumping Sqlite does not clear it. -->
     <PackageReference Include="SQLitePCLRaw.lib.e_sqlite3" Version="2.1.12" />
     <PackageReference Include="Microsoft.Data.SqlClient" Version="6.0.2" />
+    <!-- Security pin: GHSA-846f-f764-76mx / GHSA-2fmv-rv2c-59x7. Microsoft.Data.SqlClient 6.0.2
+         pulls the vulnerable System.Security.Cryptography.Pkcs 9.0.4; 10.0.13 patches it. -->
+    <PackageReference Include="System.Security.Cryptography.Pkcs" Version="10.0.13" />
     <PackageReference Include="Polly.Core" Version="8.6.6" />
   </ItemGroup>
 

src/ZB.MOM.WW.MxGateway.Worker.Tests/Bootstrap/WorkerApplicationTests.cs

@@ -1,4 +1,5 @@
 using System;
+using System.Collections.Generic;
 using System.Threading;
 using System.Threading.Tasks;
 using ZB.MOM.WW.MxGateway.Contracts;
@@ -121,6 +122,49 @@ public sealed class WorkerApplicationTests
         Assert.Equal(typeof(InvalidOperationException).FullName, entry.Fields["exception_type"]);
     }
 
+    /// <summary>
+    ///     WRK-09: the AppDomain unhandled-exception hook logs the exception type
+    ///     (never its message) and exits with the dedicated exit code.
+    /// </summary>
+    [Fact]
+    public void HandleUnhandledException_LogsTypeOnlyAndExitsWithUnhandledExceptionCode()
+    {
+        MemoryWorkerLogger logger = new();
+        int? exitCode = null;
+
+        ZB.MOM.WW.MxGateway.Worker.WorkerApplication.HandleUnhandledException(
+            logger,
+            new InvalidOperationException("password=hunter2"),
+            isTerminating: true,
+            code => exitCode = code);
+
+        Assert.Equal((int)WorkerExitCode.UnhandledException, exitCode);
+        Assert.Equal(7, (int)WorkerExitCode.UnhandledException);
+        MemoryWorkerLogEntry entry = Assert.Single(logger.Entries);
+        Assert.Equal("Error", entry.Level);
+        Assert.Equal("WorkerUnhandledException", entry.EventName);
+        Assert.Equal(typeof(InvalidOperationException).FullName, entry.Fields["exception_type"]);
+        Assert.True(entry.Fields["is_terminating"] is true);
+        Assert.DoesNotContain(
+            entry.Fields.Values,
+            value => value?.ToString()?.Contains("hunter2") == true);
+    }
+
+    /// <summary>A failing logger must not prevent the defined exit.</summary>
+    [Fact]
+    public void HandleUnhandledException_WhenLoggerThrows_StillExits()
+    {
+        int? exitCode = null;
+
+        ZB.MOM.WW.MxGateway.Worker.WorkerApplication.HandleUnhandledException(
+            new ThrowingLogger(),
+            exceptionObject: null,
+            isTerminating: false,
+            code => exitCode = code);
+
+        Assert.Equal((int)WorkerExitCode.UnhandledException, exitCode);
+    }
+
     private static string[] ValidArgs(string? protocolVersion = null)
     {
         return
@@ -171,4 +215,15 @@ public sealed class WorkerApplicationTests
             throw _exception;
         }
     }
+
+    private sealed class ThrowingLogger : IWorkerLogger
+    {
+        /// <inheritdoc />
+        public void Information(string eventName, IReadOnlyDictionary<string, object?> fields) =>
+            throw new InvalidOperationException("logger failed");
+
+        /// <inheritdoc />
+        public void Error(string eventName, IReadOnlyDictionary<string, object?> fields) =>
+            throw new InvalidOperationException("logger failed");
+    }
 }

src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/AlarmAckWriterTests.cs

@@ -84,7 +84,9 @@ public sealed class AlarmAckWriterTests
     public void Acknowledge_Twice_ReusesItemHandle()
     {
         RecordingAckServer server = new RecordingAckServer { AuthenticateResult = 5 };
-        using AlarmAckWriter writer = new AlarmAckWriter(server, new MxAccessWriteCompletionCache(), () => { }, TimeSpan.FromMilliseconds(10), "MxGateway");
+        MxAccessWriteCompletionCache cache = new MxAccessWriteCompletionCache();
+        server.OnWrite = (handle, item) => cache.Record(handle, item, Rows(MxStatusCategory.Ok, 0));
+        using AlarmAckWriter writer = new AlarmAckWriter(server, cache, () => { }, TimeSpan.FromSeconds(1), "MxGateway");
 
         writer.Acknowledge("T.A.AckMsg", "one", Alice());
         writer.Acknowledge("T.A.AckMsg", "two", Alice());
@@ -114,7 +116,9 @@ public sealed class AlarmAckWriterTests
     public void Acknowledge_WhenCachedWriteThrows_RejectsAndReAddsNextTime()
     {
         RecordingAckServer server = new RecordingAckServer { AuthenticateResult = 5 };
-        using AlarmAckWriter writer = new AlarmAckWriter(server, new MxAccessWriteCompletionCache(), () => { }, TimeSpan.FromMilliseconds(10), "MxGateway");
+        MxAccessWriteCompletionCache cache = new MxAccessWriteCompletionCache();
+        server.OnWrite = (handle, item) => cache.Record(handle, item, Rows(MxStatusCategory.Ok, 0));
+        using AlarmAckWriter writer = new AlarmAckWriter(server, cache, () => { }, TimeSpan.FromSeconds(1), "MxGateway");
         writer.Acknowledge("T.A.AckMsg", "one", Alice());
         server.WriteException = new COMException("x", unchecked((int)0x80004005));
 
@@ -141,6 +145,265 @@ public sealed class AlarmAckWriterTests
         Assert.Contains("Unregister", server.Operations);
     }
 
+    [Fact]
+    public void Acknowledge_WhenAuthenticateUserDisconnected_TearsDownAndRejects()
+    {
+        RecordingAckServer server = new RecordingAckServer
+        {
+            AuthenticateResult = 5,
+            AuthenticateException = new COMException("gone pw-secret", unchecked((int)0x80010108)),
+        };
+        using AlarmAckWriter writer = new AlarmAckWriter(server, new MxAccessWriteCompletionCache(), () => { }, TimeSpan.FromMilliseconds(10), "MxGateway");
+
+        AlarmAckResult result = writer.Acknowledge("T.A.AckMsg", "one", Alice());
+
+        Assert.Equal(AlarmAckOutcome.WriteRejected, result.Outcome);
+        Assert.Equal(unchecked((int)0x80010108), result.NativeStatus);
+        Assert.Contains("AuthenticateUser", result.Diagnostic, StringComparison.Ordinal);
+        Assert.Contains("0x80010108", result.Diagnostic, StringComparison.Ordinal);
+        Assert.DoesNotContain("pw-secret", result.Diagnostic, StringComparison.Ordinal);
+        Assert.Contains("Unregister", server.Operations);
+        Assert.DoesNotContain(server.Operations, op => op.StartsWith("AddItem", StringComparison.Ordinal));
+    }
+
+    [Fact]
+    public void Acknowledge_WhenAuthenticateUserThrowsArgumentException_ReturnsAuthenticationFailed()
+    {
+        // Live MXAccess raises E_INVALIDARG for a bad user or wrong password, which the
+        // interop surfaces as ArgumentException rather than COMException.
+        RecordingAckServer server = new RecordingAckServer
+        {
+            AuthenticateResult = 5,
+            AuthenticateException = new ArgumentException("Value does not fall within the expected range. pw-secret"),
+        };
+        using AlarmAckWriter writer = new AlarmAckWriter(server, new MxAccessWriteCompletionCache(), () => { }, TimeSpan.FromMilliseconds(10), "MxGateway");
+
+        AlarmAckResult result = writer.Acknowledge("T.A.AckMsg", "one", Alice());
+
+        Assert.Equal(AlarmAckOutcome.AuthenticationFailed, result.Outcome);
+        Assert.Equal(unchecked((int)0x80070057), result.NativeStatus);
+        Assert.DoesNotContain("pw-secret", result.Diagnostic, StringComparison.Ordinal);
+        Assert.DoesNotContain(server.Operations, op => op.StartsWith("Write", StringComparison.Ordinal));
+        Assert.DoesNotContain("Unregister", server.Operations);
+    }
+
+    [Fact]
+    public void Acknowledge_WhenAuthenticateUserThrowsArgumentNullException_Propagates()
+    {
+        // A programming error is a bug, not an operator authentication failure.
+        RecordingAckServer server = new RecordingAckServer
+        {
+            AuthenticateResult = 5,
+            AuthenticateException = new ArgumentNullException("verifyUser"),
+        };
+        using AlarmAckWriter writer = new AlarmAckWriter(server, new MxAccessWriteCompletionCache(), () => { }, TimeSpan.FromMilliseconds(10), "MxGateway");
+
+        Assert.Throws<ArgumentNullException>(() => writer.Acknowledge("T.A.AckMsg", "one", Alice()));
+    }
+
+    [Fact]
+    public void Acknowledge_WhenOwnCodeOutsideComCallThrowsArgumentException_Propagates()
+    {
+        // The pump runs between Advise and Write. An ArgumentException from it carries
+        // the same default HRESULT (0x80070057) as an interop E_INVALIDARG, so only
+        // catching at the COM call itself keeps it from being reported as WriteRejected.
+        RecordingAckServer server = new RecordingAckServer { AuthenticateResult = 5 };
+        using AlarmAckWriter writer = new AlarmAckWriter(server, new MxAccessWriteCompletionCache(), () => throw new ArgumentException("bug"), TimeSpan.FromMilliseconds(10), "MxGateway");
+
+        ArgumentException thrown = Assert.Throws<ArgumentException>(() => writer.Acknowledge("T.A.AckMsg", "one", Alice()));
+        Assert.Equal("bug", thrown.Message);
+        Assert.DoesNotContain(server.Operations, op => op.StartsWith("Write", StringComparison.Ordinal));
+    }
+
+    [Fact]
+    public void Acknowledge_WhenWriteThrowsArgumentOutOfRangeException_Propagates()
+    {
+        RecordingAckServer server = new RecordingAckServer { AuthenticateResult = 5, WriteException = new ArgumentOutOfRangeException("value") };
+        using AlarmAckWriter writer = new AlarmAckWriter(server, new MxAccessWriteCompletionCache(), () => { }, TimeSpan.FromMilliseconds(10), "MxGateway");
+
+        Assert.Throws<ArgumentOutOfRangeException>(() => writer.Acknowledge("T.A.AckMsg", "one", Alice()));
+    }
+
+    [Fact]
+    public void Acknowledge_WhenOperatorUserNameIsNull_ThrowsBeforeAuthenticating()
+    {
+        RecordingAckServer server = new RecordingAckServer { AuthenticateResult = 5 };
+        using AlarmAckWriter writer = new AlarmAckWriter(server, new MxAccessWriteCompletionCache(), () => { }, TimeSpan.FromMilliseconds(10), "MxGateway");
+
+        Assert.Throws<ArgumentException>(() => writer.Acknowledge("T.A.AckMsg", "one", new AlarmAckOperator { UserName = null! }));
+        Assert.DoesNotContain(server.Operations, op => op.StartsWith("AuthenticateUser", StringComparison.Ordinal));
+    }
+
+    [Fact]
+    public void Acknowledge_WhenWriteThrowsArgumentException_RejectsAndDropsItem()
+    {
+        RecordingAckServer server = new RecordingAckServer { AuthenticateResult = 5, WriteException = new ArgumentException("bad") };
+        using AlarmAckWriter writer = new AlarmAckWriter(server, new MxAccessWriteCompletionCache(), () => { }, TimeSpan.FromMilliseconds(10), "MxGateway");
+
+        AlarmAckResult result = writer.Acknowledge("T.A.AckMsg", "one", Alice());
+
+        Assert.Equal(AlarmAckOutcome.WriteRejected, result.Outcome);
+        Assert.Equal(unchecked((int)0x80070057), result.NativeStatus);
+        Assert.Contains("Write", result.Diagnostic, StringComparison.Ordinal);
+        Assert.Equal(new[] { 100 }, server.RemovedItemHandles);
+        Assert.DoesNotContain("Unregister", server.Operations);
+    }
+
+    [Fact]
+    public void Acknowledge_PastCacheCap_EvictsLeastRecentlyUsedItem()
+    {
+        RecordingAckServer server = new RecordingAckServer { AuthenticateResult = 5 };
+        using AlarmAckWriter writer = new AlarmAckWriter(server, new MxAccessWriteCompletionCache(), () => { }, TimeSpan.FromMilliseconds(1), "MxGateway", maxCachedItems: 2);
+
+        writer.Acknowledge("A.X.AckMsg", "1", Alice()); // handle 100
+        writer.Acknowledge("B.X.AckMsg", "2", Alice()); // handle 101
+        writer.Acknowledge("C.X.AckMsg", "3", Alice()); // handle 102, evicts A (100)
+
+        Assert.Equal(new[] { 100 }, server.RemovedItemHandles);
+        Assert.Equal(3, server.Operations.Count(op => op.StartsWith("AddItem", StringComparison.Ordinal)));
+    }
+
+    [Fact]
+    public void Acknowledge_ReusedItem_IsMostRecentlyUsedForEviction()
+    {
+        RecordingAckServer server = new RecordingAckServer { AuthenticateResult = 5 };
+        MxAccessWriteCompletionCache cache = new MxAccessWriteCompletionCache();
+        server.OnWrite = (handle, item) => cache.Record(handle, item, Rows(MxStatusCategory.Ok, 0));
+        using AlarmAckWriter writer = new AlarmAckWriter(server, cache, () => { }, TimeSpan.FromSeconds(1), "MxGateway", maxCachedItems: 2);
+
+        writer.Acknowledge("A.X.AckMsg", "1", Alice()); // handle 100
+        writer.Acknowledge("B.X.AckMsg", "2", Alice()); // handle 101
+        writer.Acknowledge("A.X.AckMsg", "3", Alice()); // reuse 100, now most recent
+        writer.Acknowledge("C.X.AckMsg", "4", Alice()); // handle 102, evicts B (101)
+
+        Assert.Equal(new[] { 101 }, server.RemovedItemHandles);
+        Assert.Equal(3, server.Operations.Count(op => op.StartsWith("AddItem", StringComparison.Ordinal)));
+        Assert.Contains("UnAdvise", server.Operations);
+    }
+
+    [Fact]
+    public void Acknowledge_AfterUnconfirmed_UsesFreshHandleSoStaleCompletionIsNotCredited()
+    {
+        RecordingAckServer server = new RecordingAckServer { AuthenticateResult = 5 };
+        MxAccessWriteCompletionCache cache = new MxAccessWriteCompletionCache();
+        using AlarmAckWriter writer = new AlarmAckWriter(server, cache, () => { }, TimeSpan.FromMilliseconds(50), "MxGateway");
+
+        AlarmAckResult first = writer.Acknowledge("T.A.AckMsg", "one", Alice()); // handle 100, no completion
+        // The first write's late OnWriteComplete (handle 100) lands while the
+        // second write is in flight; the second write's own completion never comes.
+        server.OnWrite = (handle, item) => cache.Record(handle, 100, Rows(MxStatusCategory.Ok, 0));
+        AlarmAckResult second = writer.Acknowledge("T.A.AckMsg", "two", Alice());
+
+        Assert.Equal(AlarmAckOutcome.Unconfirmed, first.Outcome);
+        Assert.Equal(AlarmAckOutcome.Unconfirmed, second.Outcome);
+        Assert.Contains(100, server.RemovedItemHandles);
+        Assert.Equal(2, server.Operations.Count(op => op.StartsWith("AddItem", StringComparison.Ordinal)));
+        Assert.Equal(new[] { 100, 101 }, server.WrittenItemHandles);
+    }
+
+    [Fact]
+    public void Acknowledge_AfterConfirmed_ReusesHandle()
+    {
+        RecordingAckServer server = new RecordingAckServer { AuthenticateResult = 5 };
+        MxAccessWriteCompletionCache cache = new MxAccessWriteCompletionCache();
+        server.OnWrite = (handle, item) => cache.Record(handle, item, Rows(MxStatusCategory.Ok, 0));
+        using AlarmAckWriter writer = new AlarmAckWriter(server, cache, () => { }, TimeSpan.FromSeconds(1), "MxGateway");
+
+        writer.Acknowledge("T.A.AckMsg", "one", Alice());
+        writer.Acknowledge("T.A.AckMsg", "two", Alice());
+
+        Assert.Equal(new[] { 100, 100 }, server.WrittenItemHandles);
+        Assert.Empty(server.RemovedItemHandles);
+    }
+
+    [Fact]
+    public void Acknowledge_AfterDirtyReAdd_ForgetsDroppedHandleCompletion()
+    {
+        RecordingAckServer server = new RecordingAckServer { AuthenticateResult = 5 };
+        MxAccessWriteCompletionCache cache = new MxAccessWriteCompletionCache();
+        using AlarmAckWriter writer = new AlarmAckWriter(server, cache, () => { }, TimeSpan.FromMilliseconds(20), "MxGateway");
+
+        writer.Acknowledge("T.A.AckMsg", "one", Alice()); // handle 100, Unconfirmed
+        cache.Record(7, 100, Rows(MxStatusCategory.Ok, 0)); // its late completion
+        Assert.Equal(1, cache.Count);
+        server.OnWrite = (handle, item) => cache.Record(handle, item, Rows(MxStatusCategory.Ok, 0));
+
+        AlarmAckResult second = writer.Acknowledge("T.A.AckMsg", "two", Alice()); // drops 100, adds 101
+
+        Assert.Equal(AlarmAckOutcome.Applied, second.Outcome);
+        Assert.Equal(1, cache.Count);
+        Assert.Equal(0UL, cache.CurrentVersion(7, 100));
+        Assert.Equal(1UL, cache.CurrentVersion(7, 101));
+    }
+
+    [Fact]
+    public void LateCompletionForDroppedHandle_DoesNotRecreateCacheEntry()
+    {
+        RecordingAckServer server = new RecordingAckServer { AuthenticateResult = 5 };
+        MxAccessWriteCompletionCache cache = new MxAccessWriteCompletionCache();
+        using AlarmAckWriter writer = new AlarmAckWriter(server, cache, () => { }, TimeSpan.FromMilliseconds(20), "MxGateway");
+        writer.Acknowledge("T.A.AckMsg", "one", Alice()); // handle 100, Unconfirmed
+        server.OnWrite = (handle, item) => writer.RecordWriteCompletion(handle, item, Rows(MxStatusCategory.Ok, 0));
+        writer.Acknowledge("T.A.AckMsg", "two", Alice()); // drops 100, adds 101, applied
+
+        writer.RecordWriteCompletion(7, 100, Rows(MxStatusCategory.Ok, 0)); // 100's late completion
+        writer.RecordWriteCompletion(8, 101, Rows(MxStatusCategory.Ok, 0)); // another registration
+
+        Assert.Equal(1, cache.Count);
+        Assert.Equal(0UL, cache.CurrentVersion(7, 100));
+        Assert.Equal(1UL, cache.CurrentVersion(7, 101));
+    }
+
+    [Fact]
+    public void Acknowledge_PastCacheCap_ForgetsEvictedHandleCompletion()
+    {
+        RecordingAckServer server = new RecordingAckServer { AuthenticateResult = 5 };
+        MxAccessWriteCompletionCache cache = new MxAccessWriteCompletionCache();
+        server.OnWrite = (handle, item) => cache.Record(handle, item, Rows(MxStatusCategory.Ok, 0));
+        using AlarmAckWriter writer = new AlarmAckWriter(server, cache, () => { }, TimeSpan.FromSeconds(1), "MxGateway", maxCachedItems: 2);
+
+        writer.Acknowledge("A.X.AckMsg", "1", Alice()); // handle 100
+        writer.Acknowledge("B.X.AckMsg", "2", Alice()); // handle 101
+        Assert.Equal(2, cache.Count);
+        writer.Acknowledge("C.X.AckMsg", "3", Alice()); // handle 102, evicts A (100)
+
+        Assert.Equal(2, cache.Count);
+        Assert.Equal(0UL, cache.CurrentVersion(7, 100));
+    }
+
+    [Fact]
+    public void Dispose_ForgetsServerCompletions()
+    {
+        RecordingAckServer server = new RecordingAckServer { AuthenticateResult = 5 };
+        MxAccessWriteCompletionCache cache = new MxAccessWriteCompletionCache();
+        server.OnWrite = (handle, item) => cache.Record(handle, item, Rows(MxStatusCategory.Ok, 0));
+        AlarmAckWriter writer = new AlarmAckWriter(server, cache, () => { }, TimeSpan.FromSeconds(1), "MxGateway");
+        writer.Acknowledge("A.X.AckMsg", "1", Alice());
+        writer.Acknowledge("B.X.AckMsg", "2", Alice());
+        cache.Record(8, 100, Rows(MxStatusCategory.Ok, 0)); // another server's entry survives
+
+        writer.Dispose();
+
+        Assert.Equal(1, cache.Count);
+        Assert.Equal(1UL, cache.CurrentVersion(8, 100));
+    }
+
+    [Fact]
+    public void Acknowledge_WhenWriteDisconnected_ForgetsServerCompletions()
+    {
+        RecordingAckServer server = new RecordingAckServer { AuthenticateResult = 5 };
+        MxAccessWriteCompletionCache cache = new MxAccessWriteCompletionCache();
+        server.OnWrite = (handle, item) => cache.Record(handle, item, Rows(MxStatusCategory.Ok, 0));
+        using AlarmAckWriter writer = new AlarmAckWriter(server, cache, () => { }, TimeSpan.FromSeconds(1), "MxGateway");
+        writer.Acknowledge("A.X.AckMsg", "1", Alice()); // handle 100 recorded
+        server.WriteException = new COMException("x", unchecked((int)0x80010108));
+
+        writer.Acknowledge("A.X.AckMsg", "2", Alice());
+
+        Assert.Contains("Unregister", server.Operations);
+        Assert.Equal(0, cache.Count);
+    }
+
     [Fact]
     public void Acknowledge_WhenNonOkCompletionHasNoDetail_ReturnsMinusOne()
     {
@@ -180,6 +443,10 @@ internal sealed class RecordingAckServer : IMxAccessServer
 
     public Action<int, int>? OnWrite { get; set; }
 
+    public List<int> RemovedItemHandles { get; } = new();
+
+    public List<int> WrittenItemHandles { get; } = new();
+
     public int Register(string clientName)
     {
         Operations.Add("Register:" + clientName);
@@ -196,7 +463,11 @@ internal sealed class RecordingAckServer : IMxAccessServer
 
     public int AddItem2(int serverHandle, string itemDefinition, string itemContext) => AddItem(serverHandle, itemDefinition);
 
-    public void RemoveItem(int serverHandle, int itemHandle) => Operations.Add("RemoveItem");
+    public void RemoveItem(int serverHandle, int itemHandle)
+    {
+        Operations.Add("RemoveItem");
+        RemovedItemHandles.Add(itemHandle);
+    }
 
     public void Advise(int serverHandle, int itemHandle)
     {
@@ -216,6 +487,7 @@ internal sealed class RecordingAckServer : IMxAccessServer
     public void Write(int serverHandle, int itemHandle, object? value, int userId)
     {
         Operations.Add("Write:" + value + ":" + userId);
+        WrittenItemHandles.Add(itemHandle);
         if (WriteException is not null)
         {
             throw WriteException;

src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/AlarmCommandHandlerTests.cs

@@ -499,6 +499,40 @@ public sealed class AlarmCommandHandlerTests
         Assert.Equal(AlarmProviderMode.Subtag, evt.OnAlarmProviderModeChanged.Mode);
     }
 
+    /// <summary>
+    ///     In auto mode the primary consumer is built before the standby. When
+    ///     building the standby throws, the primary is owned by nothing — the
+    ///     failed Subscribe never reaches the dispatcher that would dispose it —
+    ///     so the handler must dispose it rather than leak its COM object, and
+    ///     the original failure must still propagate.
+    /// </summary>
+    [Fact]
+    public void Subscribe_AutoModeWhenStandbyBuildThrows_DisposesPrimaryAndRethrows()
+    {
+        FakeConsumer primary = new FakeConsumer();
+        InvalidOperationException standbyFailure = new InvalidOperationException("standby build failed");
+        AlarmCommandHandler handler = new AlarmCommandHandler(
+            new MxAccessEventQueue(),
+            () => primary,
+            threadAffinityCheck: null,
+            comFactory: null,
+            standbyFactory: _ => throw standbyFailure);
+
+        SubscribeAlarmsCommand command = new SubscribeAlarmsCommand
+        {
+            SubscriptionExpression = @"\\HOST\Galaxy!Area",
+            ForcedMode = AlarmProviderMode.Unspecified,
+        };
+        command.WatchList.Add(NewTarget("Galaxy!Area.Tank01.Level.HiHi"));
+
+        InvalidOperationException thrown = Assert.Throws<InvalidOperationException>(
+            () => handler.Subscribe(command, "s1"));
+
+        Assert.Same(standbyFailure, thrown);
+        Assert.True(primary.Disposed, "The primary consumer built before the failing standby must be disposed.");
+        Assert.False(handler.IsSubscribed);
+    }
+
     /// <summary>An explicitly fallback-capable auto subscription builds a replaceable standby even when discovery is initially empty.</summary>
     [Fact]
     public void Subscribe_AutoModeWithEmptyWatchListAndFallbackEnabled_BuildsReplaceableStandby()

src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/AlarmRecordTransitionMapperTests.cs

@@ -102,6 +102,22 @@ public sealed class AlarmRecordTransitionMapperTests
         Assert.Equal(expected, AlarmRecordTransitionMapper.MapTransition(previous, current));
     }
 
+    /// <summary>
+    ///     RETRIGGER is reserved and never synthesized: no state pair — including
+    ///     cleared-to-active, the re-alarm shape — maps to it.
+    /// </summary>
+    [Fact]
+    public void MapTransition_ForEveryStatePair_NeverProducesRetrigger()
+    {
+        foreach (MxAlarmStateKind previous in System.Enum.GetValues(typeof(MxAlarmStateKind)))
+        {
+            foreach (MxAlarmStateKind current in System.Enum.GetValues(typeof(MxAlarmStateKind)))
+            {
+                Assert.NotEqual(AlarmTransitionKind.Retrigger, AlarmRecordTransitionMapper.MapTransition(previous, current));
+            }
+        }
+    }
+
     /// <summary>Verifies that valid XML fields assemble into correct UTC timestamp.</summary>
     [Fact]
     public void ParseTransitionTimestampUtc_WithValidXmlFields_AssemblesUtc()

src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/MxAccessSessionTests.cs

@@ -0,0 +1,591 @@
+using System;
+using System.Collections.Generic;
+using System.Diagnostics;
+using System.Linq;
+using Google.Protobuf.Collections;
+using Google.Protobuf.WellKnownTypes;
+using ZB.MOM.WW.MxGateway.Contracts.Proto;
+using ZB.MOM.WW.MxGateway.Worker.MxAccess;
+using ZB.MOM.WW.MxGateway.Worker.Tests.TestSupport;
+
+namespace ZB.MOM.WW.MxGateway.Worker.Tests.MxAccess;
+
+/// <summary>
+///     Direct tests of <see cref="MxAccessSession"/> handle-lifetime cache
+///     eviction and the phased ReadBulk snapshot path, driven through
+///     <see cref="MxAccessSession.CreateForTesting"/> with a recording fake
+///     server. Nothing here needs an STA: the session methods are plain calls,
+///     and ReadBulk's pump step is a test delegate that stands in for the COM
+///     OnDataChange dispatch by writing the value cache directly.
+/// </summary>
+public sealed class MxAccessSessionTests
+{
+    private const string ReadBulkTimeoutText = "timed out";
+
+    /// <summary>
+    ///     RemoveItem must evict the handle pair's OnWriteComplete completion
+    ///     entry as well as its last-value entry; otherwise an
+    ///     AddItem -> Write -> RemoveItem churn grows the completion cache by one
+    ///     entry per cycle for the life of the worker.
+    /// </summary>
+    [Fact]
+    public void RemoveItem_EvictsValueAndWriteCompletionEntriesForTheHandle()
+    {
+        RecordingMxAccessServer server = new();
+        CompletionCacheSink sink = new();
+        MxAccessValueCache valueCache = new();
+        using MxAccessSession session = MxAccessSession.CreateForTesting(server, sink, valueCache: valueCache);
+
+        int serverHandle = session.Register("client");
+        int keep = session.AddItem(serverHandle, "Tag.Keep");
+        int drop = session.AddItem(serverHandle, "Tag.Drop");
+        RecordCompletion(sink.WriteCompletionCache, serverHandle, keep);
+        RecordCompletion(sink.WriteCompletionCache, serverHandle, drop);
+        valueCache.Set(serverHandle, keep, BuildDataChange(serverHandle, keep, 1));
+        valueCache.Set(serverHandle, drop, BuildDataChange(serverHandle, drop, 2));
+
+        session.RemoveItem(serverHandle, drop);
+
+        Assert.Equal(1, sink.WriteCompletionCache.Count);
+        Assert.Equal(0UL, sink.WriteCompletionCache.CurrentVersion(serverHandle, drop));
+        Assert.Equal(1UL, sink.WriteCompletionCache.CurrentVersion(serverHandle, keep));
+        Assert.False(valueCache.TryGet(serverHandle, drop, out _));
+        Assert.True(valueCache.TryGet(serverHandle, keep, out _));
+    }
+
+    /// <summary>
+    ///     A long-lived writer that keeps adding, writing, and removing items
+    ///     must leave the completion cache empty — the leak this fix closes.
+    /// </summary>
+    [Fact]
+    public void AddItemWriteRemoveItemChurn_DoesNotGrowWriteCompletionCache()
+    {
+        RecordingMxAccessServer server = new();
+        CompletionCacheSink sink = new();
+        using MxAccessSession session = MxAccessSession.CreateForTesting(server, sink);
+        int serverHandle = session.Register("client");
+
+        for (int cycle = 0; cycle < 50; cycle++)
+        {
+            int itemHandle = session.AddItem(serverHandle, "Tag.Churn");
+            session.Write(serverHandle, itemHandle, cycle, userId: 0);
+
+            // Stand in for the sink's OnWriteComplete post-publish hook.
+            RecordCompletion(sink.WriteCompletionCache, serverHandle, itemHandle);
+            session.RemoveItem(serverHandle, itemHandle);
+        }
+
+        Assert.Equal(0, sink.WriteCompletionCache.Count);
+    }
+
+    /// <summary>
+    ///     Unregister drops every item under the server in one MXAccess call,
+    ///     so the per-item eviction never runs for them: both caches must be
+    ///     evicted by server handle, and other servers' entries kept.
+    /// </summary>
+    [Fact]
+    public void Unregister_EvictsValueAndWriteCompletionEntriesForThatServerOnly()
+    {
+        RecordingMxAccessServer server = new();
+        CompletionCacheSink sink = new();
+        MxAccessValueCache valueCache = new();
+        using MxAccessSession session = MxAccessSession.CreateForTesting(server, sink, valueCache: valueCache);
+
+        int doomed = session.Register("doomed");
+        int survivor = session.Register("survivor");
+        int doomedItemA = session.AddItem(doomed, "Tag.A");
+        int doomedItemB = session.AddItem(doomed, "Tag.B");
+        int survivorItem = session.AddItem(survivor, "Tag.C");
+        RecordCompletion(sink.WriteCompletionCache, doomed, doomedItemA);
+        RecordCompletion(sink.WriteCompletionCache, doomed, doomedItemB);
+        RecordCompletion(sink.WriteCompletionCache, survivor, survivorItem);
+        valueCache.Set(doomed, doomedItemA, BuildDataChange(doomed, doomedItemA, 1));
+        valueCache.Set(doomed, doomedItemB, BuildDataChange(doomed, doomedItemB, 2));
+        valueCache.Set(survivor, survivorItem, BuildDataChange(survivor, survivorItem, 3));
+
+        session.Unregister(doomed);
+
+        Assert.Equal(1, sink.WriteCompletionCache.Count);
+        Assert.Equal(1UL, sink.WriteCompletionCache.CurrentVersion(survivor, survivorItem));
+        Assert.Equal(1, valueCache.Count);
+        Assert.True(valueCache.TryGet(survivor, survivorItem, out _));
+        Assert.False(valueCache.TryGet(doomed, doomedItemA, out _));
+        Assert.False(valueCache.TryGet(doomed, doomedItemB, out _));
+    }
+
+    /// <summary>
+    ///     N uncached tags that never deliver an OnDataChange must cost one
+    ///     shared timeout, not N x timeout, while each still reports its own
+    ///     timed-out failure with its own item handle.
+    /// </summary>
+    [Fact]
+    public void ReadBulk_UncachedTagsThatNeverDeliver_WaitOneSharedTimeoutAndEachTimesOut()
+    {
+        RecordingMxAccessServer server = new();
+        using MxAccessSession session = MxAccessSession.CreateForTesting(server, new NoopEventSink());
+        int serverHandle = session.Register("client");
+        string[] tags = { "Tag.1", "Tag.2", "Tag.3", "Tag.4", "Tag.5" };
+        TimeSpan timeout = TimeSpan.FromMilliseconds(300);
+
+        Stopwatch stopwatch = Stopwatch.StartNew();
+        IReadOnlyList<BulkReadResult> results = session.ReadBulk(serverHandle, tags, timeout, () => { });
+        stopwatch.Stop();
+
+        // Serial would be >= 5 x 300 ms = 1500 ms. Allow generous slack for a
+        // loaded CI host while still failing the serial shape.
+        Assert.True(
+            stopwatch.Elapsed < TimeSpan.FromMilliseconds(1200),
+            $"ReadBulk took {stopwatch.Elapsed.TotalMilliseconds:F0} ms; expected one shared {timeout.TotalMilliseconds:F0} ms wait.");
+        Assert.True(stopwatch.Elapsed >= TimeSpan.FromMilliseconds(250));
+
+        Assert.Equal(tags.Length, results.Count);
+        HashSet<int> handles = new();
+        for (int index = 0; index < tags.Length; index++)
+        {
+            BulkReadResult result = results[index];
+            Assert.Equal(tags[index], result.TagAddress);
+            Assert.False(result.WasSuccessful);
+            Assert.False(result.WasCached);
+            Assert.Equal(serverHandle, result.ServerHandle);
+            Assert.Equal(server.HandleForTag(tags[index]), result.ItemHandle);
+            Assert.Equal(
+                $"ReadBulk timed out after {timeout.TotalMilliseconds:F0} ms waiting for first OnDataChange.",
+                result.ErrorMessage);
+            Assert.True(handles.Add(result.ItemHandle));
+        }
+
+        // Every snapshot handle is still torn down.
+        foreach (int handle in handles)
+        {
+            Assert.Contains($"UnAdvise:{serverHandle}:{handle}", server.Operations);
+            Assert.Contains($"RemoveItem:{serverHandle}:{handle}", server.Operations);
+        }
+
+        Assert.Empty(session.HandleRegistry.ItemHandles);
+    }
+
+    /// <summary>
+    ///     The phased lifecycle: every uncached tag is added and advised before
+    ///     the wait, and none is torn down until the wait is over.
+    /// </summary>
+    [Fact]
+    public void ReadBulk_AddsAndAdvisesEverySnapshotTagBeforeTearingAnyDown()
+    {
+        RecordingMxAccessServer server = new();
+        using MxAccessSession session = MxAccessSession.CreateForTesting(server, new NoopEventSink());
+        int serverHandle = session.Register("client");
+
+        session.ReadBulk(serverHandle, new[] { "Tag.A", "Tag.B", "Tag.C" }, TimeSpan.FromMilliseconds(50), () => { });
+
+        List<string> lifecycle = server.Operations.Where(op => !op.StartsWith("Register", StringComparison.Ordinal)).ToList();
+        int lastAdvise = lifecycle.FindLastIndex(op => op.StartsWith("Advise:", StringComparison.Ordinal));
+        int firstUnAdvise = lifecycle.FindIndex(op => op.StartsWith("UnAdvise:", StringComparison.Ordinal));
+        int firstRemove = lifecycle.FindIndex(op => op.StartsWith("RemoveItem:", StringComparison.Ordinal));
+        Assert.Equal(3, lifecycle.Count(op => op.StartsWith("AddItem:", StringComparison.Ordinal)));
+        Assert.Equal(3, lifecycle.Count(op => op.StartsWith("Advise:", StringComparison.Ordinal)));
+        Assert.True(lastAdvise < firstUnAdvise, string.Join(",", lifecycle));
+        Assert.True(lastAdvise < firstRemove, string.Join(",", lifecycle));
+    }
+
+    /// <summary>
+    ///     Every per-tag outcome keeps its exact pre-batching shape and request
+    ///     order in one mixed request: blank address, cached-and-advised fast
+    ///     path, a tag that delivers, an AddItem failure, an Advise failure, and
+    ///     a tag that never delivers. A delivering tag returns as soon as its
+    ///     value lands rather than waiting out the timeout for the others.
+    /// </summary>
+    [Fact]
+    public void ReadBulk_MixedOutcomes_PreservesPerTagResultsAndOrder()
+    {
+        RecordingMxAccessServer server = new();
+        server.FailAddItemFor.Add("Tag.AddFails");
+        server.FailAdviseFor.Add("Tag.AdviseFails");
+        MxAccessValueCache valueCache = new();
+        using MxAccessSession session = MxAccessSession.CreateForTesting(server, new NoopEventSink(), valueCache: valueCache);
+        int serverHandle = session.Register("client");
+
+        // Caller-owned advised tag with a cached value: served from the cache.
+        int cachedHandle = session.AddItem(serverHandle, "Tag.Cached");
+        session.Advise(serverHandle, cachedHandle);
+        valueCache.Set(serverHandle, cachedHandle, BuildDataChange(serverHandle, cachedHandle, 11));
+        int operationsBeforeRead = server.Operations.Count;
+
+        // Pump step: deliver Tag.Delivers' first OnDataChange once advised.
+        Action pumpStep = () =>
+        {
+            if (server.TryGetAdvisedHandle("Tag.Delivers", out int handle)
+                && !valueCache.TryGet(serverHandle, handle, out _))
+            {
+                valueCache.Set(serverHandle, handle, BuildDataChange(serverHandle, handle, 42));
+            }
+        };
+
+        string[] tags = { " ", "Tag.Cached", "Tag.Delivers", "Tag.AddFails", "Tag.AdviseFails", "Tag.Silent" };
+        IReadOnlyList<BulkReadResult> results = session.ReadBulk(serverHandle, tags, TimeSpan.FromMilliseconds(150), pumpStep);
+
+        Assert.Equal(tags.Length, results.Count);
+        for (int index = 0; index < tags.Length; index++)
+        {
+            Assert.Equal(tags[index], results[index].TagAddress);
+        }
+
+        Assert.False(results[0].WasSuccessful);
+        Assert.Equal(0, results[0].ItemHandle);
+        Assert.Equal("Tag address is required.", results[0].ErrorMessage);
+
+        Assert.True(results[1].WasSuccessful);
+        Assert.True(results[1].WasCached);
+        Assert.Equal(cachedHandle, results[1].ItemHandle);
+        Assert.Equal(11, results[1].Value.Int32Value);
+
+        Assert.True(results[2].WasSuccessful);
+        Assert.False(results[2].WasCached);
+        Assert.Equal(server.HandleForTag("Tag.Delivers"), results[2].ItemHandle);
+        Assert.Equal(42, results[2].Value.Int32Value);
+        Assert.Equal(192, results[2].Quality);
+
+        Assert.False(results[3].WasSuccessful);
+        Assert.Equal(0, results[3].ItemHandle);
+        Assert.Equal("AddItem failed for Tag.AddFails", results[3].ErrorMessage);
+
+        Assert.False(results[4].WasSuccessful);
+        Assert.Equal(server.HandleForTag("Tag.AdviseFails"), results[4].ItemHandle);
+        Assert.Equal("Advise failed for Tag.AdviseFails", results[4].ErrorMessage);
+
+        Assert.False(results[5].WasSuccessful);
+        Assert.Equal(server.HandleForTag("Tag.Silent"), results[5].ItemHandle);
+        Assert.Contains(ReadBulkTimeoutText, results[5].ErrorMessage, StringComparison.OrdinalIgnoreCase);
+
+        // The caller's subscription is untouched; the snapshot handles are gone
+        // (the Advise-failed one removed without an UnAdvise).
+        List<string> readOperations = server.Operations.Skip(operationsBeforeRead).ToList();
+        Assert.DoesNotContain($"UnAdvise:{serverHandle}:{cachedHandle}", readOperations);
+        Assert.DoesNotContain($"RemoveItem:{serverHandle}:{cachedHandle}", readOperations);
+        Assert.DoesNotContain($"UnAdvise:{serverHandle}:{server.HandleForTag("Tag.AdviseFails")}", readOperations);
+        Assert.Contains($"RemoveItem:{serverHandle}:{server.HandleForTag("Tag.AdviseFails")}", readOperations);
+        Assert.Contains($"UnAdvise:{serverHandle}:{server.HandleForTag("Tag.Delivers")}", readOperations);
+        Assert.Contains($"RemoveItem:{serverHandle}:{server.HandleForTag("Tag.Silent")}", readOperations);
+        RegisteredItemHandle remaining = Assert.Single(session.HandleRegistry.ItemHandles);
+        Assert.Equal(cachedHandle, remaining.ItemHandle);
+    }
+
+    /// <summary>
+    ///     When every snapshot tag delivers, the wait ends as soon as the last
+    ///     value lands instead of running out the deadline.
+    /// </summary>
+    [Fact]
+    public void ReadBulk_AllTagsDeliver_ReturnsWithoutWaitingOutTheTimeout()
+    {
+        RecordingMxAccessServer server = new();
+        MxAccessValueCache valueCache = new();
+        using MxAccessSession session = MxAccessSession.CreateForTesting(server, new NoopEventSink(), valueCache: valueCache);
+        int serverHandle = session.Register("client");
+        string[] tags = { "Tag.1", "Tag.2", "Tag.3" };
+
+        Action pumpStep = () =>
+        {
+            foreach (string tag in tags)
+            {
+                if (server.TryGetAdvisedHandle(tag, out int handle) && !valueCache.TryGet(serverHandle, handle, out _))
+                {
+                    valueCache.Set(serverHandle, handle, BuildDataChange(serverHandle, handle, handle));
+                }
+            }
+        };
+
+        Stopwatch stopwatch = Stopwatch.StartNew();
+        IReadOnlyList<BulkReadResult> results = session.ReadBulk(serverHandle, tags, TimeSpan.FromSeconds(10), pumpStep);
+        stopwatch.Stop();
+
+        Assert.True(stopwatch.Elapsed < TimeSpan.FromSeconds(5), $"ReadBulk took {stopwatch.Elapsed.TotalMilliseconds:F0} ms.");
+        Assert.All(results, result => Assert.True(result.WasSuccessful));
+        for (int index = 0; index < tags.Length; index++)
+        {
+            Assert.Equal(results[index].ItemHandle, results[index].Value.Int32Value);
+        }
+    }
+
+    /// <summary>
+    ///     Ghost entry: an OnDataChange/OnWriteComplete already queued when
+    ///     RemoveItem ran is dispatched after the eviction and recreates the
+    ///     entries. When MXAccess later reuses that item id, AddItem must drop
+    ///     them so ReadBulk's cached path cannot serve the stale value.
+    /// </summary>
+    [Fact]
+    public void AddItem_ReusingAnEvictedHandleId_DropsGhostCacheEntries()
+    {
+        RecordingMxAccessServer server = new();
+        CompletionCacheSink sink = new();
+        MxAccessValueCache valueCache = new();
+        using MxAccessSession session = MxAccessSession.CreateForTesting(server, sink, valueCache: valueCache);
+        int serverHandle = session.Register("client");
+
+        int first = session.AddItem(serverHandle, "Tag.Old");
+        session.Advise(serverHandle, first);
+        session.RemoveItem(serverHandle, first);
+
+        // Late dispatch of callbacks queued before the RemoveItem.
+        valueCache.Set(serverHandle, first, BuildDataChange(serverHandle, first, 999));
+        RecordCompletion(sink.WriteCompletionCache, serverHandle, first);
+
+        server.NextItemHandleOverride = first;
+        int reused = session.AddItem(serverHandle, "Tag.New");
+        Assert.Equal(first, reused);
+
+        Assert.False(valueCache.TryGet(serverHandle, reused, out _));
+        Assert.Equal(0UL, sink.WriteCompletionCache.CurrentVersion(serverHandle, reused));
+
+        // Once advised, ReadBulk on the new tag does not report the ghost as cached.
+        session.Advise(serverHandle, reused);
+        BulkReadResult result = Assert.Single(session.ReadBulk(serverHandle, new[] { "Tag.New" }, TimeSpan.FromMilliseconds(30), () => { }));
+        Assert.False(result.WasCached);
+        Assert.False(result.WasSuccessful);
+    }
+
+    /// <summary>
+    ///     A handle MXAccess returns while the registry still tracks it is a
+    ///     live item, so its cached value must survive the AddItem.
+    /// </summary>
+    [Fact]
+    public void AddItem_ReturningAStillRegisteredHandle_KeepsItsCachedValue()
+    {
+        RecordingMxAccessServer server = new() { ReuseHandleForSameTag = true };
+        MxAccessValueCache valueCache = new();
+        using MxAccessSession session = MxAccessSession.CreateForTesting(server, new NoopEventSink(), valueCache: valueCache);
+        int serverHandle = session.Register("client");
+        int handle = session.AddItem(serverHandle, "Tag.Live");
+        valueCache.Set(serverHandle, handle, BuildDataChange(serverHandle, handle, 5));
+
+        Assert.Equal(handle, session.AddItem(serverHandle, "Tag.Live"));
+
+        Assert.True(valueCache.TryGet(serverHandle, handle, out _));
+    }
+
+    /// <summary>
+    ///     Duplicate addresses that MXAccess resolves to one handle: the handle
+    ///     is advised, unadvised, and removed exactly once, and every duplicate
+    ///     receives the shared outcome.
+    /// </summary>
+    [Fact]
+    public void ReadBulk_DuplicateTagsResolvingToOneHandle_AdviseAndTearDownOnceAndFanOutResult()
+    {
+        RecordingMxAccessServer server = new() { ReuseHandleForSameTag = true };
+        MxAccessValueCache valueCache = new();
+        using MxAccessSession session = MxAccessSession.CreateForTesting(server, new NoopEventSink(), valueCache: valueCache);
+        int serverHandle = session.Register("client");
+
+        Action pumpStep = () =>
+        {
+            if (server.TryGetAdvisedHandle("Tag.Dup", out int handle) && !valueCache.TryGet(serverHandle, handle, out _))
+            {
+                valueCache.Set(serverHandle, handle, BuildDataChange(serverHandle, handle, 77));
+            }
+        };
+
+        IReadOnlyList<BulkReadResult> results = session.ReadBulk(
+            serverHandle,
+            new[] { "Tag.Dup", "Tag.Other", "Tag.Dup" },
+            TimeSpan.FromMilliseconds(100),
+            pumpStep);
+
+        int dupHandle = server.HandleForTag("Tag.Dup");
+        string advise = "Advise:" + serverHandle + ":" + dupHandle;
+        string unAdvise = "UnAdvise:" + serverHandle + ":" + dupHandle;
+        string remove = "RemoveItem:" + serverHandle + ":" + dupHandle;
+        Assert.True(results[0].WasSuccessful);
+        Assert.True(results[2].WasSuccessful);
+        Assert.Equal(dupHandle, results[0].ItemHandle);
+        Assert.Equal(dupHandle, results[2].ItemHandle);
+        Assert.Equal(77, results[2].Value.Int32Value);
+        Assert.Equal("Tag.Dup", results[2].TagAddress);
+        Assert.False(results[1].WasSuccessful);
+        Assert.Equal("Tag.Other", results[1].TagAddress);
+
+        Assert.Equal(1, server.Operations.Count(op => op == advise));
+        Assert.Equal(1, server.Operations.Count(op => op == unAdvise));
+        Assert.Equal(1, server.Operations.Count(op => op == remove));
+        Assert.Empty(session.HandleRegistry.ItemHandles);
+    }
+
+    /// <summary>
+    ///     Large batches pump during the add/advise and teardown phases too,
+    ///     not only inside the wait, so the STA heartbeat stays fresh.
+    /// </summary>
+    [Fact]
+    public void ReadBulk_LargeBatch_PumpsDuringAddAdviseAndTeardownPhases()
+    {
+        RecordingMxAccessServer server = new();
+        using MxAccessSession session = MxAccessSession.CreateForTesting(server, new NoopEventSink());
+        int serverHandle = session.Register("client");
+        string[] tags = Enumerable.Range(0, 3 * MxAccessSession.ReadBulkPumpInterval).Select(i => "Tag." + i).ToArray();
+        List<int> pumpAtOperation = new();
+
+        session.ReadBulk(serverHandle, tags, TimeSpan.FromMilliseconds(20), () => pumpAtOperation.Add(server.Operations.Count));
+
+        int lastAdvise = server.Operations.FindLastIndex(op => op.StartsWith("Advise:", StringComparison.Ordinal));
+        int firstUnAdvise = server.Operations.FindIndex(op => op.StartsWith("UnAdvise:", StringComparison.Ordinal));
+        Assert.Contains(pumpAtOperation, at => at < lastAdvise);
+        Assert.Contains(pumpAtOperation, at => at > firstUnAdvise);
+    }
+
+    private static void RecordCompletion(MxAccessWriteCompletionCache cache, int serverHandle, int itemHandle)
+    {
+        RepeatedField<MxStatusProxy> statuses = new() { new MxStatusProxy { Category = MxStatusCategory.Ok } };
+        cache.Record(serverHandle, itemHandle, statuses);
+    }
+
+    private static MxEvent BuildDataChange(int serverHandle, int itemHandle, int value)
+    {
+        MxEvent mxEvent = new()
+        {
+            Family = MxEventFamily.OnDataChange,
+            ServerHandle = serverHandle,
+            ItemHandle = itemHandle,
+            Quality = 192,
+            SourceTimestamp = Timestamp.FromDateTime(new DateTime(2026, 10, 22, 0, 0, 0, DateTimeKind.Utc)),
+            Value = new MxValue
+            {
+                DataType = MxDataType.Integer,
+                VariantType = "VT_I4",
+                Int32Value = value,
+            },
+            OnDataChange = new OnDataChangeEvent(),
+        };
+        mxEvent.Statuses.Add(new MxStatusProxy { Category = MxStatusCategory.Ok });
+        return mxEvent;
+    }
+
+    /// <summary>Test sink that owns a real completion cache (Attach/Detach are no-ops).</summary>
+    private sealed class CompletionCacheSink : IMxAccessEventSink, IWriteCompletionCacheProvider
+    {
+        /// <inheritdoc />
+        public MxAccessWriteCompletionCache WriteCompletionCache { get; } = new();
+
+        /// <inheritdoc />
+        public void Attach(object mxAccessComObject, string sessionId)
+        {
+        }
+
+        /// <inheritdoc />
+        public void Detach()
+        {
+        }
+    }
+
+    /// <summary>
+    ///     Fake server that hands out increasing handles, records every
+    ///     lifecycle call, and can fail AddItem or Advise for named tags.
+    /// </summary>
+    private sealed class RecordingMxAccessServer : IMxAccessServer
+    {
+        private readonly NoopMxAccessServer noop = new();
+        private readonly Dictionary<string, int> handleByTag = new(StringComparer.Ordinal);
+        private readonly Dictionary<int, string> tagByHandle = new();
+        private readonly HashSet<int> advised = new();
+        private int nextServerHandle = 10;
+        private int nextItemHandle = 100;
+
+        public List<string> Operations { get; } = new();
+
+        public HashSet<string> FailAddItemFor { get; } = new(StringComparer.Ordinal);
+
+        public HashSet<string> FailAdviseFor { get; } = new(StringComparer.Ordinal);
+
+        /// <summary>When set, the next AddItem returns this handle (simulates MXAccess reusing an id).</summary>
+        public int? NextItemHandleOverride { get; set; }
+
+        /// <summary>When true, AddItem of an already-added address returns the existing handle.</summary>
+        public bool ReuseHandleForSameTag { get; set; }
+
+        public int HandleForTag(string tag) => handleByTag[tag];
+
+        public bool TryGetAdvisedHandle(string tag, out int handle)
+        {
+            return handleByTag.TryGetValue(tag, out handle) && advised.Contains(handle);
+        }
+
+        public int Register(string clientName)
+        {
+            Operations.Add($"Register:{clientName}");
+            return nextServerHandle++;
+        }
+
+        public void Unregister(int serverHandle) => Operations.Add($"Unregister:{serverHandle}");
+
+        public int AddItem(int serverHandle, string itemDefinition)
+        {
+            Operations.Add($"AddItem:{serverHandle}:{itemDefinition}");
+            if (FailAddItemFor.Contains(itemDefinition))
+            {
+                throw new InvalidOperationException($"AddItem failed for {itemDefinition}");
+            }
+
+            int handle;
+            if (NextItemHandleOverride is int forced)
+            {
+                NextItemHandleOverride = null;
+                handle = forced;
+            }
+            else if (ReuseHandleForSameTag && handleByTag.TryGetValue(itemDefinition, out int existing))
+            {
+                handle = existing;
+            }
+            else
+            {
+                handle = nextItemHandle++;
+            }
+
+            handleByTag[itemDefinition] = handle;
+            tagByHandle[handle] = itemDefinition;
+            return handle;
+        }
+
+        public int AddItem2(int serverHandle, string itemDefinition, string itemContext) => AddItem(serverHandle, itemDefinition);
+
+        public void RemoveItem(int serverHandle, int itemHandle) => Operations.Add($"RemoveItem:{serverHandle}:{itemHandle}");
+
+        public void Advise(int serverHandle, int itemHandle)
+        {
+            Operations.Add($"Advise:{serverHandle}:{itemHandle}");
+            if (tagByHandle.TryGetValue(itemHandle, out string? tag) && FailAdviseFor.Contains(tag))
+            {
+                throw new InvalidOperationException($"Advise failed for {tag}");
+            }
+
+            advised.Add(itemHandle);
+        }
+
+        public void UnAdvise(int serverHandle, int itemHandle)
+        {
+            Operations.Add($"UnAdvise:{serverHandle}:{itemHandle}");
+            advised.Remove(itemHandle);
+        }
+
+        public void AdviseSupervisory(int serverHandle, int itemHandle) => Advise(serverHandle, itemHandle);
+
+        public int AddBufferedItem(int serverHandle, string itemDefinition, string itemContext) => AddItem(serverHandle, itemDefinition);
+
+        public void SetBufferedUpdateInterval(int serverHandle, int updateIntervalMilliseconds) =>
+            noop.SetBufferedUpdateInterval(serverHandle, updateIntervalMilliseconds);
+
+        public object Suspend(int serverHandle, int itemHandle) => noop.Suspend(serverHandle, itemHandle);
+
+        public object Activate(int serverHandle, int itemHandle) => noop.Activate(serverHandle, itemHandle);
+
+        public void Write(int serverHandle, int itemHandle, object? value, int userId) =>
+            Operations.Add($"Write:{serverHandle}:{itemHandle}");
+
+        public void Write2(int serverHandle, int itemHandle, object? value, object? timestampValue, int userId) =>
+            Operations.Add($"Write2:{serverHandle}:{itemHandle}");
+
+        public void WriteSecured(int serverHandle, int itemHandle, int currentUserId, int verifierUserId, object? value) =>
+            Operations.Add($"WriteSecured:{serverHandle}:{itemHandle}");
+
+        public void WriteSecured2(int serverHandle, int itemHandle, int currentUserId, int verifierUserId, object? value, object? timestampValue) =>
+            Operations.Add($"WriteSecured2:{serverHandle}:{itemHandle}");
+
+        public int AuthenticateUser(int serverHandle, string verifyUser, string verifyUserPassword) => 0;
+
+        public int ArchestrAUserToId(int serverHandle, string userIdGuid) => 0;
+    }
+}

src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/MxAccessStaSessionTests.cs

@@ -1,4 +1,5 @@
 using System;
+using System.Collections.Concurrent;
 using System.Collections.Generic;
 using System.Runtime.InteropServices;
 using System.Threading;
@@ -573,6 +574,181 @@ public sealed class MxAccessStaSessionTests
             "Expected StaCallInProgress to clear once the alarm poll returned.");
     }
 
+    /// <summary>
+    /// A transient PollOnce failure must not fault the whole session: with a
+    /// failure threshold of 3, a poll pattern that never fails three times in
+    /// a row (fail, fail, succeed, ...) keeps the session healthy, and each
+    /// transient failure is logged with type/HRESULT only — never the
+    /// exception message.
+    /// </summary>
+    /// <returns>A task that represents the asynchronous operation.</returns>
+    [Fact]
+    public async Task RunAlarmPollLoop_TransientFailuresBelowThreshold_DoNotFaultSession()
+    {
+        FakeAlarmCommandHandler handler = new()
+        {
+            PollExceptionFactory = call => call % 3 == 0
+                ? null
+                : new COMException("secret-tag-name GetXmlCurrentAlarms2 failed.", unchecked((int)0x80004005)),
+        };
+        ConcurrentQueue<string> warnings = new();
+        using StaRuntime runtime = CreateRuntime();
+        MxAccessEventQueue eventQueue = new();
+        using MxAccessStaSession session = new(
+            runtime,
+            new FakeMxAccessComObjectFactory(),
+            new FakeMxAccessEventSink(),
+            eventQueue,
+            (_eq, _affinity, _comFactory) => handler);
+        session.AlarmPollInterval = TimeSpan.FromMilliseconds(10);
+        session.AlarmPollFailureThreshold = 3;
+        session.AlarmPollWarningSink = warnings.Enqueue;
+
+        await session.StartAsync("session-1", workerProcessId: 1);
+
+        DateTime deadline = DateTime.UtcNow.AddSeconds(10);
+        while (handler.PollCount < 12 && !eventQueue.IsFaulted && DateTime.UtcNow < deadline)
+        {
+            await Task.Delay(10);
+        }
+
+        Assert.False(eventQueue.IsFaulted, "Transient alarm-poll failures below the threshold must not fault the session.");
+        Assert.True(handler.PollCount >= 12, $"Expected the poll loop to keep polling; saw {handler.PollCount} polls.");
+        Assert.NotEmpty(warnings);
+        Assert.All(warnings, line =>
+        {
+            Assert.Contains("event=AlarmPollFailed", line, StringComparison.Ordinal);
+            Assert.Contains("hresult=0x80004005", line, StringComparison.Ordinal);
+            Assert.DoesNotContain("secret-tag-name", line, StringComparison.Ordinal);
+        });
+    }
+
+    /// <summary>
+    /// A run of failures reaching the threshold with no success in between
+    /// faults the session exactly once and stops the loop: two warnings, then
+    /// the fault on the third consecutive failure, then no further polls.
+    /// </summary>
+    /// <returns>A task that represents the asynchronous operation.</returns>
+    [Fact]
+    public async Task RunAlarmPollLoop_ConsecutiveFailuresReachThreshold_FaultsAndStops()
+    {
+        FakeAlarmCommandHandler handler = new()
+        {
+            PollException = new COMException("GetXmlCurrentAlarms2 failed.", unchecked((int)0x80004005)),
+        };
+        ConcurrentQueue<string> warnings = new();
+        using StaRuntime runtime = CreateRuntime();
+        MxAccessEventQueue eventQueue = new();
+        using MxAccessStaSession session = new(
+            runtime,
+            new FakeMxAccessComObjectFactory(),
+            new FakeMxAccessEventSink(),
+            eventQueue,
+            (_eq, _affinity, _comFactory) => handler);
+        session.AlarmPollInterval = TimeSpan.FromMilliseconds(10);
+        session.AlarmPollFailureThreshold = 3;
+        session.AlarmPollWarningSink = warnings.Enqueue;
+
+        await session.StartAsync("session-1", workerProcessId: 1);
+
+        DateTime deadline = DateTime.UtcNow.AddSeconds(10);
+        while (!eventQueue.IsFaulted && DateTime.UtcNow < deadline)
+        {
+            await Task.Delay(10);
+        }
+
+        Assert.True(eventQueue.IsFaulted);
+        await Task.Delay(200);
+        Assert.Equal(3, handler.PollCount);
+        Assert.Equal(2, warnings.Count);
+        WorkerFault? fault = session.DrainFault();
+        Assert.NotNull(fault);
+        Assert.Equal(WorkerFaultCategory.MxaccessEventConversionFailed, fault!.Category);
+    }
+
+    /// <summary>
+    /// WRK-02: when the STA loop dies after the session started, the death is
+    /// recorded as a StaHung fault on the event queue — the path the IPC drain
+    /// loop turns into a WorkerFault frame and a terminated pipe session.
+    /// </summary>
+    /// <returns>A task that represents the asynchronous operation.</returns>
+    [Fact]
+    public async Task StaLoopDeath_AfterStart_RecordsStaHungFaultOnEventQueue()
+    {
+        ControllableMessagePump pump = new();
+        using StaRuntime runtime = new(new NoopComApartmentInitializer(), pump, TimeSpan.FromMilliseconds(25));
+        MxAccessEventQueue eventQueue = new();
+        using MxAccessStaSession session = new(
+            runtime,
+            new FakeMxAccessComObjectFactory(),
+            new FakeMxAccessEventSink(),
+            eventQueue);
+
+        await session.StartAsync("session-1", workerProcessId: 1);
+        Assert.False(eventQueue.IsFaulted);
+
+        pump.ThrowOnNextPump(new InvalidOperationException("secret-tag-name pump failed."));
+
+        DateTime deadline = DateTime.UtcNow.AddSeconds(5);
+        while (!eventQueue.IsFaulted && DateTime.UtcNow < deadline)
+        {
+            await Task.Delay(10);
+        }
+
+        Assert.True(eventQueue.IsFaulted, "Expected the STA loop death to fault the event queue.");
+        WorkerFault? fault = session.DrainFault();
+        Assert.NotNull(fault);
+        Assert.Equal(WorkerFaultCategory.StaHung, fault!.Category);
+        Assert.Equal(typeof(InvalidOperationException).FullName, fault.ExceptionType);
+        Assert.Contains("STA loop terminated", fault.DiagnosticMessage, StringComparison.Ordinal);
+        Assert.Contains("exceptionType=System.InvalidOperationException", fault.DiagnosticMessage, StringComparison.Ordinal);
+        Assert.Contains("hresult=0x", fault.DiagnosticMessage, StringComparison.Ordinal);
+        Assert.DoesNotContain("secret-tag-name", fault.DiagnosticMessage, StringComparison.Ordinal);
+        Assert.DoesNotContain("secret-tag-name", fault.ProtocolStatus.Message, StringComparison.Ordinal);
+        Assert.Equal(ProtocolStatusCode.WorkerUnavailable, fault.ProtocolStatus.Code);
+    }
+
+    /// <summary>
+    /// After the STA loop dies, the alarm poll's fast-failed InvokeAsync must
+    /// not record a second, mislabelled alarm-poll fault: once the StaHung
+    /// fault is drained, nothing else is recorded.
+    /// </summary>
+    /// <returns>A task that represents the asynchronous operation.</returns>
+    [Fact]
+    public async Task RunAlarmPollLoop_AfterStaLoopFault_DoesNotRecordAlarmPollFault()
+    {
+        ControllableMessagePump pump = new();
+        FakeAlarmCommandHandler handler = new();
+        using StaRuntime runtime = new(new NoopComApartmentInitializer(), pump, TimeSpan.FromMilliseconds(25));
+        MxAccessEventQueue eventQueue = new();
+        using MxAccessStaSession session = new(
+            runtime,
+            new FakeMxAccessComObjectFactory(),
+            new FakeMxAccessEventSink(),
+            eventQueue,
+            (_eq, _affinity, _comFactory) => handler);
+        session.AlarmPollInterval = TimeSpan.FromMilliseconds(10);
+        session.AlarmPollFailureThreshold = 1;
+        session.AlarmPollWarningSink = _ => { };
+
+        await session.StartAsync("session-1", workerProcessId: 1);
+        pump.ThrowOnNextPump(new InvalidOperationException("pump failed"));
+
+        DateTime deadline = DateTime.UtcNow.AddSeconds(5);
+        while (!eventQueue.IsFaulted && DateTime.UtcNow < deadline)
+        {
+            await Task.Delay(10);
+        }
+
+        WorkerFault? first = session.DrainFault();
+        Assert.NotNull(first);
+        Assert.Equal(WorkerFaultCategory.StaHung, first!.Category);
+
+        // Give the poll loop several ticks to (wrongly) record its own fault.
+        await Task.Delay(300);
+        Assert.Null(session.DrainFault());
+    }
+
     /// <summary>
     /// The STA-affinity guard throws when an
     /// IMxAccessAlarmConsumer call is attempted off the thread that created
@@ -621,6 +797,9 @@ public sealed class MxAccessStaSessionTests
         /// <summary>Exception thrown by PollOnce; null to succeed.</summary>
         public Exception? PollException { get; set; }
 
+        /// <summary>Per-call exception factory (1-based call number); null result succeeds.</summary>
+        public Func<int, Exception?>? PollExceptionFactory { get; set; }
+
         /// <summary>When set, <see cref="PollOnce"/> blocks until <see cref="ReleasePoll"/> is called.</summary>
         public bool BlockPoll { get; set; }
 
@@ -699,6 +878,22 @@ public sealed class MxAccessStaSessionTests
             {
                 throw PollException;
             }
+
+            Func<int, Exception?>? factory = PollExceptionFactory;
+            if (factory is not null)
+            {
+                int callNumber;
+                lock (gate)
+                {
+                    callNumber = pollCount;
+                }
+
+                Exception? exception = factory(callNumber);
+                if (exception is not null)
+                {
+                    throw exception;
+                }
+            }
         }
 
         /// <inheritdoc />

src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/MxAccessWriteCompletionCacheTests.cs

@@ -183,6 +183,39 @@ public sealed class MxAccessWriteCompletionCacheTests
         Assert.Equal(77, Assert.Single(statuses).Detail);
     }
 
+    /// <summary>Verifies that Remove forgets only the given handle pair.</summary>
+    [Fact]
+    public void Remove_ForgetsOnlyThatHandlePair()
+    {
+        MxAccessWriteCompletionCache cache = new();
+        cache.Record(7, 21, BuildStatuses(detail: 1));
+        cache.Record(7, 22, BuildStatuses(detail: 2));
+
+        cache.Remove(7, 21);
+        cache.Remove(7, 99); // unknown pair is a no-op
+
+        Assert.Equal(1, cache.Count);
+        Assert.Equal(0UL, cache.CurrentVersion(7, 21));
+        Assert.Equal(1UL, cache.CurrentVersion(7, 22));
+    }
+
+    /// <summary>Verifies that RemoveServer forgets every pair of that server and no other.</summary>
+    [Fact]
+    public void RemoveServer_ForgetsOnlyThatServersPairs()
+    {
+        MxAccessWriteCompletionCache cache = new();
+        cache.Record(7, 21, BuildStatuses(detail: 1));
+        cache.Record(7, -5, BuildStatuses(detail: 2));
+        cache.Record(8, 21, BuildStatuses(detail: 3));
+        cache.Record(-7, 21, BuildStatuses(detail: 4));
+
+        cache.RemoveServer(7);
+
+        Assert.Equal(2, cache.Count);
+        Assert.Equal(1UL, cache.CurrentVersion(8, 21));
+        Assert.Equal(1UL, cache.CurrentVersion(-7, 21));
+    }
+
     private static RepeatedField<MxStatusProxy> BuildStatuses(int detail)
     {
         return new RepeatedField<MxStatusProxy>

src/ZB.MOM.WW.MxGateway.Worker.Tests/Probes/ItemHandleReuseProbeTests.cs

@@ -0,0 +1,185 @@
+// ItemHandleReuseProbeTests.cs
+//
+// Probe for issue #166: does MXAccess hand out a new item handle number when the
+// same item is removed and re-added on one LMXProxyServer registration? The
+// AlarmAckWriter's Unconfirmed mitigation (drop the item, re-add it, wait on the
+// fresh handle) relies on the number changing, because OnWriteComplete carries
+// only the (server, item) handle pair.
+//
+// The probe only adds, advises and removes items. It never writes, so it does
+// not acknowledge the alarm or create Historian rows.
+//
+// HOW TO RUN (dev rig with MXAccess installed and the DEV Galaxy running):
+//   1. Remove (or set to null) the Skip parameter on the [Fact] below.
+//   2. dotnet test src/ZB.MOM.WW.MxGateway.Worker.Tests/ZB.MOM.WW.MxGateway.Worker.Tests.csproj
+//        -p:Platform=x86 --filter FullyQualifiedName~ItemHandleReuseProbeTests
+//        --logger "console;verbosity=detailed"
+//   3. Read the logged handles. Results are recorded in docs/AlarmProbeFindings.md.
+//
+// RESULT (windev, 2026-10-20): within one registration item handles increase
+// monotonically and are never reused (1..15 across add/remove/re-add of the same
+// item, with and without Advise, interleaved with another item, and while held
+// twice). Item numbering restarts at 1 after Unregister + Register, and a fresh
+// COM object is handed server handle 1 again.
+
+using System;
+using System.Collections.Generic;
+using System.Runtime.InteropServices;
+using System.Threading;
+using ArchestrA.MxAccess;
+using ZB.MOM.WW.MxGateway.Worker.MxAccess;
+using Xunit.Abstractions;
+
+namespace ZB.MOM.WW.MxGateway.Worker.Tests.Probes;
+
+/// <summary>
+///     Live probe that logs the item handles MXAccess returns when the same
+///     <c>AckMsg</c> item is removed and re-added, to decide whether a late
+///     <c>OnWriteComplete</c> for a dropped handle could match a re-added one.
+/// </summary>
+public sealed class ItemHandleReuseProbeTests
+{
+    private const string AckItem = "TestMachine_001.TestAlarm001.AckMsg";
+    private const string OtherItem = "TestMachine_001.TestAlarm001.Acked";
+    private const int Rounds = 5;
+
+    private readonly ITestOutputHelper output;
+    private readonly List<string> log = new List<string>();
+
+    // Item handles returned on the first registration, in the order they were handed out.
+    private readonly List<int> firstRegistrationHandles = new List<int>();
+
+    /// <summary>Initializes a new instance of the <see cref="ItemHandleReuseProbeTests"/> class.</summary>
+    /// <param name="output">Test output helper for logging.</param>
+    public ItemHandleReuseProbeTests(ITestOutputHelper output)
+    {
+        this.output = output;
+    }
+
+    /// <summary>Adds, removes and re-adds the AckMsg item on a fresh registration and logs every handle.</summary>
+    [Fact(Skip = "Live dev-rig probe (issue #166) — flip Skip=null on a host with MXAccess and the DEV Galaxy running.")]
+    public void RemoveThenReAdd_LogsItemHandles()
+    {
+        Exception? threadException = null;
+        Thread thread = new Thread(() =>
+        {
+            try { RunProbe(); }
+            catch (Exception ex) { threadException = ex; }
+        });
+        thread.SetApartmentState(ApartmentState.STA);
+        thread.Start();
+        thread.Join();
+
+        foreach (string line in log)
+        {
+            output.WriteLine(line);
+        }
+
+        if (threadException is not null)
+        {
+            throw new InvalidOperationException("Probe failed: " + threadException.Message, threadException);
+        }
+
+        // The ack writer's drop-and-re-add mitigation depends on this: within one
+        // registration a handle number is never handed out twice.
+        Assert.Equal((2 * Rounds) + 5, firstRegistrationHandles.Count);
+        for (int i = 1; i < firstRegistrationHandles.Count; i++)
+        {
+            Assert.True(
+                firstRegistrationHandles[i] > firstRegistrationHandles[i - 1],
+                $"Item handle {firstRegistrationHandles[i]} did not increase past {firstRegistrationHandles[i - 1]}; MXAccess reused or reordered item handles within one registration.");
+        }
+    }
+
+    private void RunProbe()
+    {
+        LMXProxyServerClass com = new LMXProxyServerClass();
+        try
+        {
+            IMxAccessServer mx = new MxAccessComServer(com);
+            int server = mx.Register("MxGatewayProbe166");
+            Log($"Register -> server handle {server}");
+
+            // A: the AlarmAckWriter drop path (Advise, then UnAdvise + RemoveItem).
+            for (int i = 0; i < Rounds; i++)
+            {
+                int h = Held(mx.AddItem(server, AckItem));
+                mx.Advise(server, h);
+                Log($"A{i}: AddItem+Advise {AckItem} -> {h}");
+                mx.UnAdvise(server, h);
+                mx.RemoveItem(server, h);
+                Log($"A{i}: UnAdvise+RemoveItem {h}");
+            }
+
+            // B: RemoveItem without Advise.
+            for (int i = 0; i < Rounds; i++)
+            {
+                int h = Held(mx.AddItem(server, AckItem));
+                Log($"B{i}: AddItem {AckItem} -> {h}");
+                mx.RemoveItem(server, h);
+                Log($"B{i}: RemoveItem {h}");
+            }
+
+            // C: interleave another item to see whether a freed number is handed to any item.
+            int c1 = Held(mx.AddItem(server, AckItem));
+            Log($"C: AddItem {AckItem} -> {c1}");
+            mx.RemoveItem(server, c1);
+            Log($"C: RemoveItem {c1}");
+            int c2 = Held(mx.AddItem(server, OtherItem));
+            Log($"C: AddItem {OtherItem} -> {c2}");
+            int c3 = Held(mx.AddItem(server, AckItem));
+            Log($"C: AddItem {AckItem} -> {c3}");
+            mx.RemoveItem(server, c2);
+            mx.RemoveItem(server, c3);
+            Log($"C: RemoveItem {c2}, {c3}");
+
+            // D: the same item added twice while the first is still held.
+            int d1 = Held(mx.AddItem(server, AckItem));
+            int d2 = Held(mx.AddItem(server, AckItem));
+            Log($"D: AddItem {AckItem} twice while held -> {d1}, {d2}");
+            mx.RemoveItem(server, d1);
+            mx.RemoveItem(server, d2);
+
+            mx.Unregister(server);
+            Log($"Unregister {server}");
+
+            // E: the TearDownServer path: re-register on the same COM object and add again.
+            int server2 = mx.Register("MxGatewayProbe166");
+            int e1 = mx.AddItem(server2, AckItem);
+            Log($"E: re-Register -> server handle {server2}; AddItem {AckItem} -> {e1}");
+            mx.RemoveItem(server2, e1);
+            mx.Unregister(server2);
+        }
+        finally
+        {
+            Marshal.FinalReleaseComObject(com);
+        }
+
+        // F: a fresh COM object, as AlarmAckWriter creates after a teardown.
+        LMXProxyServerClass com2 = new LMXProxyServerClass();
+        try
+        {
+            IMxAccessServer mx2 = new MxAccessComServer(com2);
+            int server3 = mx2.Register("MxGatewayProbe166");
+            int f1 = mx2.AddItem(server3, AckItem);
+            Log($"F: new COM object Register -> server handle {server3}; AddItem {AckItem} -> {f1}");
+            mx2.RemoveItem(server3, f1);
+            mx2.Unregister(server3);
+        }
+        finally
+        {
+            Marshal.FinalReleaseComObject(com2);
+        }
+    }
+
+    private int Held(int itemHandle)
+    {
+        firstRegistrationHandles.Add(itemHandle);
+        return itemHandle;
+    }
+
+    private void Log(string line)
+    {
+        log.Add(DateTime.Now.ToString("HH:mm:ss.fff") + " " + line);
+    }
+}

src/ZB.MOM.WW.MxGateway.Worker.Tests/Sta/StaRuntimeTests.cs

@@ -2,6 +2,7 @@ using System;
 using System.Threading;
 using System.Threading.Tasks;
 using ZB.MOM.WW.MxGateway.Worker.Sta;
+using ZB.MOM.WW.MxGateway.Worker.Tests.TestSupport;
 
 namespace ZB.MOM.WW.MxGateway.Worker.Tests.Sta;
 
@@ -155,6 +156,145 @@ public sealed class StaRuntimeTests
         Assert.Contains("shutting down", exception.Message);
     }
 
+    /// <summary>
+    ///     A backlog of queued work items must not starve inbound COM
+    ///     callbacks: the loop pumps Windows messages between items rather
+    ///     than only after draining the whole queue. A gate item holds the STA
+    ///     while two more items queue behind it; once released, the second
+    ///     queued item must observe a pump that happened after the first.
+    /// </summary>
+    /// <returns>A task that represents the asynchronous operation.</returns>
+    [Fact]
+    public async Task ProcessQueuedCommands_PumpsMessagesBetweenQueuedWorkItems()
+    {
+        ControllableMessagePump pump = new();
+        using StaRuntime runtime = new(new RecordingComApartmentInitializer(), pump, TimeSpan.FromSeconds(30));
+        runtime.Start();
+
+        using ManualResetEventSlim gateEntered = new(false);
+        using ManualResetEventSlim releaseGate = new(false);
+        Task gate = runtime.InvokeAsync(() =>
+        {
+            gateEntered.Set();
+            releaseGate.Wait(TimeSpan.FromSeconds(10));
+        });
+        Assert.True(gateEntered.Wait(TimeSpan.FromSeconds(5)));
+
+        Task<int> first = runtime.InvokeAsync(() => pump.PumpCount);
+        Task<int> second = runtime.InvokeAsync(() => pump.PumpCount);
+        releaseGate.Set();
+
+        await gate;
+        int pumpsSeenByFirst = await first;
+        int pumpsSeenBySecond = await second;
+
+        Assert.True(
+            pumpsSeenBySecond > pumpsSeenByFirst,
+            $"Expected a pump between queued work items (first saw {pumpsSeenByFirst}, second saw {pumpsSeenBySecond}).");
+    }
+
+    /// <summary>
+    ///     WRK-02: an exception that kills the STA loop after a successful
+    ///     <see cref="StaRuntime.Start"/> must be surfaced — recorded on
+    ///     <see cref="StaRuntime.LoopFault"/>, handed to the registered fault
+    ///     handler, and turned into fast failures for every later
+    ///     <see cref="StaRuntime.InvokeAsync{T}(Func{T}, CancellationToken)"/> —
+    ///     rather than parked in a startup field nothing reads.
+    /// </summary>
+    /// <returns>A task that represents the asynchronous operation.</returns>
+    [Fact]
+    public async Task LoopException_AfterStart_IsReportedToHandlerAndFailsLaterCommands()
+    {
+        ControllableMessagePump pump = new();
+        RecordingComApartmentInitializer initializer = new();
+        using StaRuntime runtime = new(initializer, pump, TimeSpan.FromMilliseconds(25));
+        runtime.Start();
+
+        TaskCompletionSource<Exception> reported = new(TaskCreationOptions.RunContinuationsAsynchronously);
+        runtime.RegisterLoopFaultHandler(exception => reported.TrySetResult(exception));
+        Assert.Null(runtime.LoopFault);
+
+        InvalidOperationException loopException = new("pump failed");
+        pump.ThrowOnNextPump(loopException);
+
+        Task completed = await Task.WhenAny(reported.Task, Task.Delay(TimeSpan.FromSeconds(5)));
+        Assert.Same(reported.Task, completed);
+        Assert.Same(loopException, await reported.Task);
+        Assert.Same(loopException, runtime.LoopFault);
+
+        InvalidOperationException rejected = await Assert.ThrowsAsync<InvalidOperationException>(
+            () => runtime.InvokeAsync(() => 1));
+        Assert.IsNotType<StaRuntimeShutdownException>(rejected);
+        Assert.Same(loopException, rejected.InnerException);
+
+        // The dying thread still runs its apartment cleanup.
+        Assert.True(runtime.Shutdown(TimeSpan.FromSeconds(5)));
+        Assert.False(runtime.IsRunning);
+        Assert.Equal(1, initializer.UninitializeCount);
+    }
+
+    /// <summary>
+    ///     A handler registered after the loop already died still receives the
+    ///     fault, so the window between Start and registration loses nothing.
+    /// </summary>
+    /// <returns>A task that represents the asynchronous operation.</returns>
+    [Fact]
+    public async Task RegisterLoopFaultHandler_AfterLoopDied_InvokesHandlerImmediately()
+    {
+        ControllableMessagePump pump = new();
+        using StaRuntime runtime = new(new RecordingComApartmentInitializer(), pump, TimeSpan.FromMilliseconds(25));
+        runtime.Start();
+        InvalidOperationException loopException = new("pump failed");
+        pump.ThrowOnNextPump(loopException);
+
+        DateTime deadline = DateTime.UtcNow.AddSeconds(5);
+        while (runtime.LoopFault is null && DateTime.UtcNow < deadline)
+        {
+            await Task.Delay(10);
+        }
+
+        Exception? seen = null;
+        runtime.RegisterLoopFaultHandler(exception => seen = exception);
+
+        Assert.Same(loopException, seen);
+    }
+
+    /// <summary>A requested shutdown is not a loop fault.</summary>
+    [Fact]
+    public void Shutdown_DoesNotRecordLoopFault()
+    {
+        using StaRuntime runtime = new(new RecordingComApartmentInitializer(), new ControllableMessagePump(), TimeSpan.FromMilliseconds(25));
+        runtime.Start();
+        bool invoked = false;
+        runtime.RegisterLoopFaultHandler(_ => invoked = true);
+
+        Assert.True(runtime.Shutdown(TimeSpan.FromSeconds(5)));
+
+        Assert.Null(runtime.LoopFault);
+        Assert.False(invoked);
+    }
+
+    /// <summary>
+    ///     A loop that throws while a requested shutdown is in progress is
+    ///     being torn down on purpose: the loop-fault handler must not run.
+    /// </summary>
+    [Fact]
+    public void LoopException_DuringRequestedShutdown_DoesNotInvokeHandler()
+    {
+        ControllableMessagePump pump = new();
+        using StaRuntime runtime = new(new RecordingComApartmentInitializer(), pump, TimeSpan.FromSeconds(30));
+        runtime.Start();
+        bool invoked = false;
+        runtime.RegisterLoopFaultHandler(_ => invoked = true);
+
+        // The loop is parked in the 30 s idle wait; Shutdown wakes it and the
+        // very next pump throws with shutdownRequested already set.
+        pump.ThrowOnNextPump(new InvalidOperationException("pump failed during shutdown"));
+        Assert.True(runtime.Shutdown(TimeSpan.FromSeconds(5)));
+
+        Assert.False(invoked);
+    }
+
     private static StaRuntime CreateRuntime(RecordingComApartmentInitializer initializer)
     {
         return new StaRuntime(

src/ZB.MOM.WW.MxGateway.Worker.Tests/TestSupport/ControllableMessagePump.cs

@@ -0,0 +1,47 @@
+using System;
+using System.Threading;
+using ZB.MOM.WW.MxGateway.Worker.Sta;
+
+namespace ZB.MOM.WW.MxGateway.Worker.Tests.TestSupport;
+
+/// <summary>
+/// Test <see cref="IStaMessagePump"/> that counts pump calls and can be armed
+/// to throw from the next pump, so tests can observe the STA loop's pump
+/// interleaving and drive a post-start loop death without a real Win32
+/// message queue. The idle wait blocks on the command wake event exactly
+/// like the production pump's <c>MsgWaitForMultipleObjectsEx</c> does.
+/// </summary>
+internal sealed class ControllableMessagePump : IStaMessagePump
+{
+    private int pumpCount;
+    private Exception? throwOnNextPump;
+
+    /// <summary>Gets the number of <see cref="PumpPendingMessages"/> calls so far.</summary>
+    public int PumpCount => Volatile.Read(ref pumpCount);
+
+    /// <summary>Arms the pump to throw <paramref name="exception"/> from its next pump call.</summary>
+    /// <param name="exception">The exception to throw once.</param>
+    public void ThrowOnNextPump(Exception exception)
+    {
+        Volatile.Write(ref throwOnNextPump, exception);
+    }
+
+    /// <inheritdoc />
+    public void WaitForWorkOrMessages(WaitHandle commandWakeEvent, TimeSpan timeout)
+    {
+        commandWakeEvent.WaitOne(timeout);
+    }
+
+    /// <inheritdoc />
+    public int PumpPendingMessages()
+    {
+        Interlocked.Increment(ref pumpCount);
+        Exception? exception = Interlocked.Exchange(ref throwOnNextPump, null);
+        if (exception is not null)
+        {
+            throw exception;
+        }
+
+        return 0;
+    }
+}

src/ZB.MOM.WW.MxGateway.Worker/Bootstrap/WorkerExitCode.cs

@@ -9,4 +9,13 @@ public enum WorkerExitCode
     MissingNonce = 4,
     PipeConnectionFailed = 5,
     ProtocolViolation = 6,
+
+    /// <summary>
+    ///     An exception escaped to <c>AppDomain.UnhandledException</c> on some
+    ///     thread (a background task, a COM callback thread, a timer) rather
+    ///     than through <c>WorkerApplication.Run</c>'s own catch blocks. The
+    ///     worker logs it and exits with this code instead of the CLR's crash
+    ///     exit code.
+    /// </summary>
+    UnhandledException = 7,
 }

src/ZB.MOM.WW.MxGateway.Worker/Ipc/WorkerPipeSession.cs

@@ -76,7 +76,7 @@ public sealed class WorkerPipeSession
             options,
             () => Process.GetCurrentProcess().Id,
             new WorkerPipeSessionOptions(),
-            () => new MxAccessStaSession((eq, affinity, comFactory) => new AlarmCommandHandler(eq, () => new WnWrapAlarmConsumer(), affinity, comFactory, standbyFactory: null)),
+            () => new MxAccessStaSession((eq, affinity, comFactory, pump) => new AlarmCommandHandler(eq, () => new WnWrapAlarmConsumer(), affinity, comFactory, standbyFactory: null, ackPumpStep: pump)),
             logger,
             stream)
     {
@@ -98,7 +98,7 @@ public sealed class WorkerPipeSession
             options,
             processIdProvider,
             new WorkerPipeSessionOptions(),
-            () => new MxAccessStaSession((eq, affinity, comFactory) => new AlarmCommandHandler(eq, () => new WnWrapAlarmConsumer(), affinity, comFactory, standbyFactory: null)),
+            () => new MxAccessStaSession((eq, affinity, comFactory, pump) => new AlarmCommandHandler(eq, () => new WnWrapAlarmConsumer(), affinity, comFactory, standbyFactory: null, ackPumpStep: pump)),
             logger: null)
     {
     }
@@ -1585,7 +1585,7 @@ public sealed class WorkerPipeSession
         // parameterless CompleteStartupHandshakeAsync is used without a
         // prior factory call.
         _runtimeSession ??= new MxAccessStaSession(
-            (eq, affinity, comFactory) => new AlarmCommandHandler(eq, () => new WnWrapAlarmConsumer(), affinity, comFactory, standbyFactory: null));
+            (eq, affinity, comFactory, pump) => new AlarmCommandHandler(eq, () => new WnWrapAlarmConsumer(), affinity, comFactory, standbyFactory: null, ackPumpStep: pump));
         IWorkerRuntimeSession session = _runtimeSession;
         try
         {

src/ZB.MOM.WW.MxGateway.Worker/MxAccess/AlarmAckWriter.cs

@@ -26,13 +26,30 @@ public sealed class AlarmAckWriter : IAlarmAckWriter
     /// <summary>Default MXAccess client name when none is configured.</summary>
     public const string DefaultClientName = "MxGateway";
 
+    /// <summary>
+    ///     Default cap on cached ack item handles. Each cached item holds an
+    ///     advised MXAccess item for the worker's life, so the cache is bounded
+    ///     (least-recently-used eviction) rather than growing with every
+    ///     distinct alarm ever acknowledged.
+    /// </summary>
+    public const int DefaultMaxCachedItems = 256;
+
     private readonly IMxAccessComObjectFactory? factory;
     private readonly MxAccessWriteCompletionCache completionCache;
     private readonly Action pumpStep;
     private readonly TimeSpan completionTimeout;
     private readonly string clientName;
     private readonly MxStatusProxyConverter statusConverter = new MxStatusProxyConverter();
-    private readonly Dictionary<string, int> itemHandles = new Dictionary<string, int>(StringComparer.OrdinalIgnoreCase);
+    private readonly int maxCachedItems;
+    private readonly Dictionary<string, LinkedListNode<AckItem>> itemHandles = new Dictionary<string, LinkedListNode<AckItem>>(StringComparer.OrdinalIgnoreCase);
+
+    // Most-recently-used first; the tail is the eviction candidate.
+    private readonly LinkedList<AckItem> itemLru = new LinkedList<AckItem>();
+
+    // Item handles currently added on this registration. A completion for any
+    // other handle (a dropped one whose OnWriteComplete arrives late) is ignored
+    // so it cannot recreate a completion-cache entry for a dead handle.
+    private readonly HashSet<int> heldItemHandles = new HashSet<int>();
     private object? comObject;
     private LMXProxyServerClass? comEventSource;
     private IMxAccessServer? server;
@@ -54,6 +71,7 @@ public sealed class AlarmAckWriter : IAlarmAckWriter
         this.pumpStep = pumpStep ?? throw new ArgumentNullException(nameof(pumpStep));
         this.completionTimeout = completionTimeout;
         this.clientName = string.IsNullOrWhiteSpace(clientName) ? DefaultClientName : clientName!;
+        maxCachedItems = DefaultMaxCachedItems;
         completionCache = new MxAccessWriteCompletionCache();
     }
 
@@ -63,8 +81,11 @@ public sealed class AlarmAckWriter : IAlarmAckWriter
         MxAccessWriteCompletionCache completionCache,
         Action pumpStep,
         TimeSpan completionTimeout,
-        string clientName)
+        string clientName,
+        int maxCachedItems = DefaultMaxCachedItems)
     {
+        if (maxCachedItems < 1) throw new ArgumentOutOfRangeException(nameof(maxCachedItems));
+        this.maxCachedItems = maxCachedItems;
         this.server = server ?? throw new ArgumentNullException(nameof(server));
         this.completionCache = completionCache ?? throw new ArgumentNullException(nameof(completionCache));
         this.pumpStep = pumpStep ?? throw new ArgumentNullException(nameof(pumpStep));
@@ -88,19 +109,36 @@ public sealed class AlarmAckWriter : IAlarmAckWriter
         if (string.IsNullOrWhiteSpace(itemAddress)) throw new ArgumentException("Ack item address is required.", nameof(itemAddress));
         if (ackOperator is null) throw new ArgumentNullException(nameof(ackOperator));
 
+        if (ackOperator.UserName is null) throw new ArgumentException("Operator user name is required.", nameof(ackOperator));
+        string verifyUser = ackOperator.VerifyUser;
+        string password = ackOperator.Password ?? string.Empty;
+        string commentValue = comment ?? string.Empty;
+
         IMxAccessServer mx = EnsureRegistered();
 
         int userId;
         try
         {
-            userId = mx.AuthenticateUser(serverHandle, ackOperator.VerifyUser, ackOperator.Password ?? string.Empty);
+            userId = Interop("AuthenticateUser", () => mx.AuthenticateUser(serverHandle, verifyUser, password));
         }
-        catch (COMException ex)
+        catch (AckInteropException ex)
         {
+            int authHresult = ex.HResult;
+            if (IsServerGone(authHresult))
+            {
+                // The registration is dead, not the credentials: tear down so the
+                // next ack re-registers, and report it as a write-path failure.
+                TearDownServer();
+                return AlarmAckResult.Create(
+                    AlarmAckOutcome.WriteRejected,
+                    authHresult,
+                    $"AckMsg AuthenticateUser failed (HRESULT 0x{authHresult:X8}).");
+            }
+
             return AlarmAckResult.Create(
                 AlarmAckOutcome.AuthenticationFailed,
                 ex.HResult,
-                $"Operator authentication failed for '{ackOperator.VerifyUser}' (HRESULT 0x{ex.HResult:X8}).");
+                $"Operator authentication failed for '{verifyUser}' (HRESULT 0x{ex.HResult:X8}).");
         }
 
         if (userId == 0)
@@ -108,36 +146,59 @@ public sealed class AlarmAckWriter : IAlarmAckWriter
             return AlarmAckResult.Create(
                 AlarmAckOutcome.AuthenticationFailed,
                 -1,
-                $"Operator authentication failed for '{ackOperator.VerifyUser}' (no user id returned).");
+                $"Operator authentication failed for '{verifyUser}' (no user id returned).");
         }
 
         ulong baselineVersion = 0;
         int itemHandle = 0;
-        bool cached = itemHandles.TryGetValue(itemAddress, out itemHandle);
-        string step = "AddItem";
+        LinkedListNode<AckItem>? node;
+        bool cached = itemHandles.TryGetValue(itemAddress, out node);
         try
         {
-            if (!cached)
+            if (cached && node!.Value.Dirty)
             {
-                itemHandle = mx.AddItem(serverHandle, itemAddress);
-                itemHandles[itemAddress] = itemHandle;
-                step = "Advise";
+                // The previous ack to this item ended Unconfirmed, so its
+                // OnWriteComplete may still arrive. Completions are keyed by
+                // (server, item) handle with no per-write id, so a late one would
+                // be credited to this write. Drop the item and re-add it so this
+                // write waits on a fresh handle the stale completion cannot match.
+                // That holds because MXAccess never reuses an item handle number
+                // within one registration: the live probe (ItemHandleReuseProbeTests,
+                // windev 2026-10-20) saw strictly increasing handles across repeated
+                // remove/re-add of the same item. Numbering restarts only on a new
+                // registration, and TearDownServer detaches this writer's
+                // OnWriteComplete handler and clears the server's completion entries
+                // first, so the old registration's completions cannot reach it.
+                DropItem(mx, node);
+                cached = false;
+            }
+
+            if (cached)
+            {
+                itemHandle = node!.Value.Handle;
+                itemLru.Remove(node);
+                itemLru.AddFirst(node);
+            }
+            else
+            {
+                itemHandle = Interop("AddItem", () => mx.AddItem(serverHandle, itemAddress));
+                node = CacheItem(mx, itemAddress, itemHandle);
+                int added = itemHandle;
                 // MXAccess rejects writes to added-but-unadvised items with E_INVALIDARG.
-                mx.Advise(serverHandle, itemHandle);
+                Interop("Advise", () => mx.Advise(serverHandle, added));
             }
 
             pumpStep();
-            step = "Write";
             baselineVersion = completionCache.CurrentVersion(serverHandle, itemHandle);
-            mx.Write(serverHandle, itemHandle, comment ?? string.Empty, userId);
+            int target = itemHandle;
+            Interop("Write", () => mx.Write(serverHandle, target, commentValue, userId));
         }
-        catch (COMException ex)
+        catch (AckInteropException ex)
         {
             int hresult = ex.HResult;
-            if (itemHandles.Remove(itemAddress))
+            if (itemHandles.TryGetValue(itemAddress, out LinkedListNode<AckItem>? failed))
             {
-                try { mx.UnAdvise(serverHandle, itemHandle); } catch { /* best effort */ }
-                try { mx.RemoveItem(serverHandle, itemHandle); } catch { /* best effort */ }
+                DropItem(mx, failed!);
             }
 
             if (IsServerGone(hresult))
@@ -149,7 +210,7 @@ public sealed class AlarmAckWriter : IAlarmAckWriter
             return AlarmAckResult.Create(
                 AlarmAckOutcome.WriteRejected,
                 hresult,
-                $"AckMsg {step} failed (HRESULT 0x{hresult:X8}).");
+                $"AckMsg {ex.Step} failed (HRESULT 0x{hresult:X8}).");
         }
 
         ulong baseline = baselineVersion;
@@ -162,6 +223,7 @@ public sealed class AlarmAckWriter : IAlarmAckWriter
                 pumpStep,
                 out RepeatedField<MxStatusProxy> statuses))
         {
+            node!.Value.Dirty = true;
             return AlarmAckResult.Create(
                 AlarmAckOutcome.Unconfirmed,
                 0,
@@ -192,6 +254,54 @@ public sealed class AlarmAckWriter : IAlarmAckWriter
         TearDownServer();
     }
 
+    /// <summary>
+    ///     Runs one MXAccess COM call and converts a failure the interop raised
+    ///     for its HRESULT into <see cref="AckInteropException"/>. Only the COM
+    ///     call itself runs inside the filter, so an exception thrown by this
+    ///     writer's own code (argument checks, caching, the pump) propagates as
+    ///     a bug instead of being reported as an MXAccess failure.
+    /// </summary>
+    /// <remarks>
+    ///     The runtime maps well-known HRESULTs to typed exceptions and the rest
+    ///     to COMException; each carries the HRESULT in Exception.HResult. The
+    ///     ArgumentException arm is needed because a wrong password or unknown
+    ///     user makes <c>AuthenticateUser</c> throw ArgumentException with
+    ///     E_INVALIDARG (0x80070057), observed live on windev on 2026-10-19.
+    ///     ArgumentNullException and ArgumentOutOfRangeException are excluded:
+    ///     the interop does not raise them for an HRESULT, so one escaping the
+    ///     call is an argument bug in the adapter, not an MXAccess failure.
+    /// </remarks>
+    private static T Interop<T>(string step, Func<T> call)
+    {
+        try
+        {
+            return call();
+        }
+        catch (Exception ex) when (IsInteropFailure(ex))
+        {
+            throw new AckInteropException(step, ex);
+        }
+    }
+
+    private static void Interop(string step, Action call)
+    {
+        try
+        {
+            call();
+        }
+        catch (Exception ex) when (IsInteropFailure(ex))
+        {
+            throw new AckInteropException(step, ex);
+        }
+    }
+
+    private static bool IsInteropFailure(Exception ex) =>
+        ex is ExternalException
+        || ex is UnauthorizedAccessException
+        || (ex is ArgumentException
+            && ex is not ArgumentNullException
+            && ex is not ArgumentOutOfRangeException);
+
     private static bool IsServerGone(int hresult) =>
         hresult == unchecked((int)0x80010108) // RPC_E_DISCONNECTED
         || hresult == unchecked((int)0x800706BA) // RPC_S_SERVER_UNAVAILABLE
@@ -208,16 +318,21 @@ public sealed class AlarmAckWriter : IAlarmAckWriter
         IMxAccessServer? mx = server;
         if (mx is not null)
         {
-            foreach (int itemHandle in itemHandles.Values)
+            foreach (AckItem item in itemLru)
             {
-                try { mx.UnAdvise(serverHandle, itemHandle); } catch { /* best effort */ }
-                try { mx.RemoveItem(serverHandle, itemHandle); } catch { /* best effort */ }
+                try { mx.UnAdvise(serverHandle, item.Handle); } catch { /* best effort */ }
+                try { mx.RemoveItem(serverHandle, item.Handle); } catch { /* best effort */ }
             }
 
             try { mx.Unregister(serverHandle); } catch { /* best effort */ }
         }
 
+        // The registration is gone, so no completion can arrive for its handles.
+        completionCache.RemoveServer(serverHandle);
+
         itemHandles.Clear();
+        itemLru.Clear();
+        heldItemHandles.Clear();
         server = null;
         comEventSource = null;
         object? toRelease = comObject;
@@ -228,6 +343,34 @@ public sealed class AlarmAckWriter : IAlarmAckWriter
         }
     }
 
+    /// <summary>Caches a newly added item as most recently used, evicting the least recently used past the cap.</summary>
+    private LinkedListNode<AckItem> CacheItem(IMxAccessServer mx, string itemAddress, int itemHandle)
+    {
+        while (itemLru.Count >= maxCachedItems && itemLru.Last is { } leastRecent)
+        {
+            DropItem(mx, leastRecent);
+        }
+
+        LinkedListNode<AckItem> node = itemLru.AddFirst(new AckItem(itemAddress, itemHandle));
+        itemHandles[itemAddress] = node;
+        heldItemHandles.Add(itemHandle);
+        return node;
+    }
+
+    /// <summary>Removes a cached item and best-effort releases its MXAccess item.</summary>
+    private void DropItem(IMxAccessServer mx, LinkedListNode<AckItem> node)
+    {
+        itemHandles.Remove(node.Value.Address);
+        itemLru.Remove(node);
+        heldItemHandles.Remove(node.Value.Handle);
+        try { mx.UnAdvise(serverHandle, node.Value.Handle); } catch { /* best effort */ }
+        try { mx.RemoveItem(serverHandle, node.Value.Handle); } catch { /* best effort */ }
+
+        // Without this the completion cache keeps one entry per dropped handle
+        // (each Unconfirmed re-add and each eviction) for the worker's life.
+        completionCache.Remove(serverHandle, node.Value.Handle);
+    }
+
     private IMxAccessServer EnsureRegistered()
     {
         if (server is not null) return server;
@@ -274,6 +417,53 @@ public sealed class AlarmAckWriter : IAlarmAckWriter
     {
         RepeatedField<MxStatusProxy> rows = new RepeatedField<MxStatusProxy>();
         rows.AddRange(statusConverter.ConvertMany(pVars));
-        completionCache.Record(hLMXServerHandle, phItemHandle, rows);
+        RecordWriteCompletion(hLMXServerHandle, phItemHandle, rows);
+    }
+
+    /// <summary>
+    ///     Records a write completion only for an item handle this writer
+    ///     currently holds on its live registration. A late completion for a
+    ///     dropped handle would otherwise recreate the cache entry that
+    ///     <see cref="DropItem"/> removed. Internal so tests can stand in for
+    ///     the COM event.
+    /// </summary>
+    internal void RecordWriteCompletion(int completionServerHandle, int itemHandle, RepeatedField<MxStatusProxy> rows)
+    {
+        if (server is null || completionServerHandle != serverHandle || !heldItemHandles.Contains(itemHandle))
+        {
+            return;
+        }
+
+        completionCache.Record(completionServerHandle, itemHandle, rows);
+    }
+
+    /// <summary>An MXAccess failure raised by one COM call, carrying that call's HRESULT.</summary>
+    private sealed class AckInteropException : Exception
+    {
+        public AckInteropException(string step, Exception inner)
+            : base("MXAccess " + step + " failed.", inner)
+        {
+            Step = step;
+            HResult = inner.HResult;
+        }
+
+        public string Step { get; }
+    }
+
+    /// <summary>A cached ack item. Plain class (not a record) for the net48 worker target.</summary>
+    private sealed class AckItem
+    {
+        public AckItem(string address, int handle)
+        {
+            Address = address;
+            Handle = handle;
+        }
+
+        public string Address { get; }
+
+        public int Handle { get; }
+
+        /// <summary>Set when the last write to this handle ended Unconfirmed; forces a fresh handle next time.</summary>
+        public bool Dirty { get; set; }
     }
 }

src/ZB.MOM.WW.MxGateway.Worker/MxAccess/AlarmCommandHandler.cs

@@ -41,6 +41,7 @@ public sealed class AlarmCommandHandler : IAlarmCommandHandler
     private readonly IMxAccessComObjectFactory? comFactory;
     private readonly Func<IReadOnlyList<AlarmSubtagTarget>, IMxAccessAlarmConsumer>? standbyFactory;
     private readonly Func<IAlarmAckWriter>? ackWriterFactory;
+    private readonly Action? ackPumpStep;
     private readonly MxAccessEventMapper mapper = new MxAccessEventMapper();
     private readonly object syncRoot = new object();
     private readonly IAlarmAckWriter deferredAckWriter;
@@ -131,13 +132,21 @@ public sealed class AlarmCommandHandler : IAlarmCommandHandler
     ///     unsubscribe/resubscribe, and is disposed with the handler. Tests
     ///     inject a fake.
     /// </param>
+    /// <param name="ackPumpStep">
+    ///     Pump step the default <see cref="AlarmAckWriter"/> drives while it waits
+    ///     for OnWriteComplete. Production passes the STA runtime's
+    ///     <c>StaRuntime.PumpPendingMessages</c>, which also refreshes the STA
+    ///     watchdog heartbeat; when <see langword="null"/> a bare
+    ///     <see cref="StaMessagePump"/> is used (no heartbeat refresh).
+    /// </param>
     public AlarmCommandHandler(
         MxAccessEventQueue eventQueue,
         Func<IMxAccessAlarmConsumer> consumerFactory,
         Action? threadAffinityCheck,
         IMxAccessComObjectFactory? comFactory,
         Func<IReadOnlyList<AlarmSubtagTarget>, IMxAccessAlarmConsumer>? standbyFactory,
-        Func<IAlarmAckWriter>? ackWriterFactory = null)
+        Func<IAlarmAckWriter>? ackWriterFactory = null,
+        Action? ackPumpStep = null)
     {
         this.eventQueue = eventQueue ?? throw new ArgumentNullException(nameof(eventQueue));
         this.consumerFactory = consumerFactory ?? throw new ArgumentNullException(nameof(consumerFactory));
@@ -145,6 +154,7 @@ public sealed class AlarmCommandHandler : IAlarmCommandHandler
         this.comFactory = comFactory;
         this.standbyFactory = standbyFactory;
         this.ackWriterFactory = ackWriterFactory;
+        this.ackPumpStep = ackPumpStep;
         this.deferredAckWriter = new DeferredAckWriter(this);
     }
 
@@ -236,13 +246,32 @@ public sealed class AlarmCommandHandler : IAlarmCommandHandler
         {
             IMxAccessAlarmConsumer primary = consumerFactory()
                 ?? throw new InvalidOperationException("Alarm consumer factory returned null.");
-            IMxAccessAlarmConsumer standby = BuildStandby(watchList);
-            AlarmFailoverConfig? failoverConfig = command.Failover;
-            FailoverSettings settings = new FailoverSettings(
-                failoverConfig?.ConsecutiveFailureThreshold ?? 3,
-                failoverConfig?.FailbackProbeIntervalSeconds ?? 30,
-                failoverConfig?.FailbackStableProbes ?? 3);
-            return new FailoverAlarmConsumer(primary, standby, settings);
+            IMxAccessAlarmConsumer? standby = null;
+            try
+            {
+                standby = BuildStandby(watchList);
+                AlarmFailoverConfig? failoverConfig = command.Failover;
+                FailoverSettings settings = new FailoverSettings(
+                    failoverConfig?.ConsecutiveFailureThreshold ?? 3,
+                    failoverConfig?.FailbackProbeIntervalSeconds ?? 30,
+                    failoverConfig?.FailbackStableProbes ?? 3);
+                return new FailoverAlarmConsumer(primary, standby, settings);
+            }
+            catch
+            {
+                // The primary (and a standby that was built before the
+                // composite constructor threw) is owned by nothing yet: the
+                // failed Subscribe never reaches the dispatcher that would
+                // dispose it, so release it here or its wnwrap COM object
+                // leaks for the life of the worker.
+                if (standby is not null)
+                {
+                    try { standby.Dispose(); } catch { /* swallow — preserve the build failure */ }
+                }
+
+                try { primary.Dispose(); } catch { /* swallow — preserve the build failure */ }
+                throw;
+            }
         }
 
         // Alarmmgr, or Unspecified with an empty watch list — primary only.
@@ -399,12 +428,14 @@ public sealed class AlarmCommandHandler : IAlarmCommandHandler
                     "Alarm acknowledgement requires an IMxAccessComObjectFactory; the alarm command handler was constructed without one.");
             }
 
-            // StaMessagePump.PumpPendingMessages is PeekMessage/DispatchMessage on
-            // the calling thread — the STA, since acks run there — which is what
-            // delivers the writer's OnWriteComplete callback.
+            // The pump step is PeekMessage/DispatchMessage on the calling thread —
+            // the STA, since acks run there — which is what delivers the writer's
+            // OnWriteComplete callback. The injected StaRuntime pump also marks STA
+            // activity so a long completion wait does not read as a hung STA.
+            Action pump = ackPumpStep ?? (() => new StaMessagePump().PumpPendingMessages());
             ackWriter = new AlarmAckWriter(
                 comFactory,
-                () => new StaMessagePump().PumpPendingMessages(),
+                pump,
                 MxAccessStaSession.ResolveWriteCompletionTimeout(),
                 AlarmAckWriter.ResolveClientName());
             return ackWriter;

src/ZB.MOM.WW.MxGateway.Worker/MxAccess/AlarmRecordTransitionMapper.cs

@@ -49,6 +49,17 @@ public static class AlarmRecordTransitionMapper
     ///         <item><description><c>previous=*Rtn</c> + <c>current=*Alm</c> → Raise (re-trigger after clear).</description></item>
     ///         <item><description>Anything else → Unspecified (no proto kind to emit).</description></item>
     ///     </list>
+    ///     <para>
+    ///         <see cref="AlarmTransitionKind.Retrigger"/> is never returned. It is
+    ///         reserved in the contract, but neither alarm source gives the worker
+    ///         a native re-alarm signal: the alarmmgr XML carries only the four
+    ///         <c>*_ALM</c>/<c>*_RTN</c> states, so a return-to-alarm is
+    ///         indistinguishable from a fresh raise and is reported as Raise, and
+    ///         the subtag source observes no re-alarm counter. Deliberately not
+    ///         synthesized (the gateway forwards only transitions the source
+    ///         reports — see docs/AlarmClientDiscovery.md, "Retrigger
+    ///         transitions").
+    ///     </para>
     /// </summary>
     /// <param name="previous">The previous alarm state kind.</param>
     /// <param name="current">The current alarm state kind.</param>

src/ZB.MOM.WW.MxGateway.Worker/MxAccess/MxAccessSession.cs

@@ -222,6 +222,13 @@ public sealed class MxAccessSession : IDisposable
 
         mxAccessServer.Unregister(serverHandle);
         handleRegistry.UnregisterServerHandle(serverHandle);
+        // Unregister drops every item under the server in one call, so the
+        // per-item evictions in RemoveItem never run for them. Evict both
+        // per-handle caches by server so a long-lived worker that registers
+        // and unregisters clients does not accumulate one entry per item it
+        // ever wrote or read, and a reused server handle id starts clean.
+        valueCache.RemoveServer(serverHandle);
+        writeCompletionCache.RemoveServer(serverHandle);
     }
 
     /// <summary>Adds an item to an MXAccess server and returns the item handle.</summary>
@@ -235,6 +242,7 @@ public sealed class MxAccessSession : IDisposable
         ThrowIfDisposed();
 
         int itemHandle = mxAccessServer.AddItem(serverHandle, itemDefinition);
+        EvictStaleHandleState(serverHandle, itemHandle);
         handleRegistry.RegisterItemHandle(
             serverHandle,
             itemHandle,
@@ -258,6 +266,7 @@ public sealed class MxAccessSession : IDisposable
         ThrowIfDisposed();
 
         int itemHandle = mxAccessServer.AddItem2(serverHandle, itemDefinition, itemContext);
+        EvictStaleHandleState(serverHandle, itemHandle);
         handleRegistry.RegisterItemHandle(
             serverHandle,
             itemHandle,
@@ -268,6 +277,30 @@ public sealed class MxAccessSession : IDisposable
         return itemHandle;
     }
 
+    /// <summary>
+    ///     Drops cached per-handle state left over from a previous lifetime of
+    ///     a handle id MXAccess just handed out again. RemoveItem and Unregister
+    ///     already evict, but an OnDataChange or OnWriteComplete that was
+    ///     queued on the STA when they ran is dispatched afterwards (the loop
+    ///     pumps between work items) and recreates the entry. Without this, a
+    ///     reused id would let ReadBulk's cached fast path serve that stale
+    ///     value as <c>was_cached = true</c>. A handle the registry still
+    ///     tracks is a live item (MXAccess returned an existing handle), so its
+    ///     current entries are kept.
+    /// </summary>
+    private void EvictStaleHandleState(
+        int serverHandle,
+        int itemHandle)
+    {
+        if (handleRegistry.ContainsItemHandle(serverHandle, itemHandle))
+        {
+            return;
+        }
+
+        valueCache.Remove(serverHandle, itemHandle);
+        writeCompletionCache.Remove(serverHandle, itemHandle);
+    }
+
     /// <summary>Removes an item from an MXAccess server.</summary>
     /// <param name="serverHandle">Handle returned by the worker.</param>
     /// <param name="itemHandle">Handle returned by the worker.</param>
@@ -283,6 +316,11 @@ public sealed class MxAccessSession : IDisposable
         // same handle id (which MXAccess may reuse) does not serve a stale
         // OnDataChange snapshot from the previous lifetime.
         valueCache.Remove(serverHandle, itemHandle);
+        // Same for the OnWriteComplete completion entry: the event sink records
+        // one per written handle pair, so without this an AddItem -> Write ->
+        // RemoveItem churn grows the cache by one entry per cycle for the life
+        // of the worker.
+        writeCompletionCache.Remove(serverHandle, itemHandle);
     }
 
     /// <summary>Advises on item changes with plain subscription.</summary>
@@ -397,6 +435,7 @@ public sealed class MxAccessSession : IDisposable
         ThrowIfDisposed();
 
         int itemHandle = mxAccessServer.AddBufferedItem(serverHandle, itemDefinition, itemContext);
+        EvictStaleHandleState(serverHandle, itemHandle);
         handleRegistry.RegisterItemHandle(
             serverHandle,
             itemHandle,
@@ -870,14 +909,28 @@ public sealed class MxAccessSession : IDisposable
     ///     OnDataChange value if the tag is already advised AND a cached value
     ///     exists (no subscription side effects); otherwise takes the AddItem
     ///     + Advise + wait + UnAdvise + RemoveItem snapshot lifecycle itself.
-    ///     <paramref name="timeout"/> bounds the wait per-tag in the snapshot
-    ///     case; <paramref name="pumpStep"/> is invoked on every poll
-    ///     iteration so the worker's STA can dispatch the incoming MXAccess
-    ///     message that carries the value.
+    ///     <paramref name="timeout"/> bounds the wait for every snapshot tag;
+    ///     <paramref name="pumpStep"/> is invoked on every poll iteration so the
+    ///     worker's STA can dispatch the incoming MXAccess message that carries
+    ///     the value.
     /// </summary>
+    /// <remarks>
+    ///     The snapshot tags are processed in phases rather than one at a time:
+    ///     every cache probe runs first, then every uncached tag is added and
+    ///     advised, then one wait with a single shared deadline collects the
+    ///     first OnDataChange per handle, then every snapshot handle is torn
+    ///     down. A serial lifecycle held the STA for up to N x timeout when N
+    ///     tags never produced a value; the phased form holds it for at most
+    ///     one timeout. Each tag's result is still its own: a tag that never
+    ///     delivered fails with the same timed-out diagnostic and its own item
+    ///     handle, an AddItem or Advise failure fails only that tag, and results
+    ///     keep request order. The cache probes run before any snapshot AddItem
+    ///     so a later tag can never be served from an earlier tag's own
+    ///     snapshot subscription.
+    /// </remarks>
     /// <param name="serverHandle">The MXAccess server handle.</param>
     /// <param name="tagAddresses">The tag addresses to read.</param>
-    /// <param name="timeout">The timeout per tag.</param>
+    /// <param name="timeout">The timeout bounding the shared snapshot wait.</param>
     /// <param name="pumpStep">Action invoked on each poll iteration.</param>
     /// <returns>The per-tag read results, in the order the tags were provided.</returns>
     public IReadOnlyList<BulkReadResult> ReadBulk(
@@ -897,91 +950,225 @@ public sealed class MxAccessSession : IDisposable
             throw new ArgumentNullException(nameof(pumpStep));
         }
 
-        List<BulkReadResult> results = new(tagAddresses.Count);
-        foreach (string? tagAddress in tagAddresses)
+        int tagCount = tagAddresses.Count;
+        BulkReadResult?[] results = new BulkReadResult?[tagCount];
+        List<SnapshotRead> snapshots = new();
+
+        // Phase 1: validation and the cached-and-advised fast path. A tag
+        // already advised by the caller with a cached value is answered without
+        // touching the existing subscription — the caller didn't create it, so
+        // ReadBulk must not tear it down.
+        for (int index = 0; index < tagCount; index++)
         {
+            string? tagAddress = tagAddresses[index];
             if (string.IsNullOrWhiteSpace(tagAddress))
             {
-                results.Add(FailedRead(serverHandle, tagAddress ?? string.Empty, itemHandle: 0, wasCached: false, "Tag address is required."));
+                results[index] = FailedRead(serverHandle, tagAddress ?? string.Empty, itemHandle: 0, wasCached: false, "Tag address is required.");
                 continue;
             }
 
-            results.Add(ReadOneTag(serverHandle, tagAddress, timeout, pumpStep));
+            if (TryGetCachedReadFor(serverHandle, tagAddress!, out int cachedItemHandle, out MxAccessValueCache.CachedValue cachedValue))
+            {
+                results[index] = SucceededRead(serverHandle, tagAddress!, cachedItemHandle, wasCached: true, cachedValue);
+                continue;
+            }
+
+            snapshots.Add(new SnapshotRead(index, tagAddress!));
         }
 
-        return results;
+        if (snapshots.Count > 0)
+        {
+            try
+            {
+                ReadSnapshots(serverHandle, snapshots, results, timeout, pumpStep);
+            }
+            finally
+            {
+                TearDownSnapshots(serverHandle, snapshots, pumpStep);
+            }
+        }
+
+        List<BulkReadResult> ordered = new(tagCount);
+        for (int index = 0; index < tagCount; index++)
+        {
+            ordered.Add(results[index]!);
+        }
+
+        return ordered;
     }
 
-    private BulkReadResult ReadOneTag(
+    /// <summary>
+    ///     Number of snapshot items ReadBulk adds/advises (phase 2) or tears
+    ///     down (phase 4) between pump steps. Those phases make one or two COM
+    ///     calls per tag without waiting, so a large batch would otherwise hold
+    ///     the STA with no pump and no activity mark for its whole length; a
+    ///     pump every <see cref="ReadBulkPumpInterval"/> items keeps inbound
+    ///     callbacks flowing and keeps the STA heartbeat fresh, so a big batch
+    ///     cannot drift toward <c>HeartbeatStuckCeiling</c>.
+    /// </summary>
+    internal const int ReadBulkPumpInterval = 16;
+
+    private void ReadSnapshots(
         int serverHandle,
-        string tagAddress,
+        List<SnapshotRead> snapshots,
+        BulkReadResult?[] results,
         TimeSpan timeout,
         Action pumpStep)
     {
-        // 1. Cached-and-advised fast path: scan the registry for a live item
-        //    handle matching this tag and check whether the value cache has a
-        //    payload for it. If so, return the cached value without touching
-        //    the existing subscription — the caller didn't create it, so
-        //    ReadBulk must not tear it down.
-        if (TryGetCachedReadFor(serverHandle, tagAddress, out int cachedItemHandle, out MxAccessValueCache.CachedValue cachedValue))
+        // Phase 2: reserve our own item handle and advise for every uncached
+        // tag. A per-tag AddItem/Advise failure fails only that tag. When
+        // MXAccess hands back a handle an earlier tag in this batch already
+        // advised (a duplicate address), that tag shares the earlier tag's
+        // subscription instead of advising the handle a second time, and gets
+        // the same outcome fanned out to it after the wait.
+        List<SnapshotRead> waiting = new(snapshots.Count);
+        Dictionary<int, SnapshotRead> advisedByHandle = new();
+        List<SnapshotRead> sharing = new();
+        for (int position = 0; position < snapshots.Count; position++)
         {
-            return SucceededRead(
-                serverHandle,
-                tagAddress,
-                cachedItemHandle,
-                wasCached: true,
-                cachedValue);
+            SnapshotRead snapshot = snapshots[position];
+            try
+            {
+                if (position > 0 && position % ReadBulkPumpInterval == 0)
+                {
+                    pumpStep();
+                }
+
+                snapshot.ItemHandle = AddItem(serverHandle, snapshot.TagAddress);
+                if (advisedByHandle.TryGetValue(snapshot.ItemHandle, out SnapshotRead? owner))
+                {
+                    snapshot.SharedWith = owner;
+                    sharing.Add(snapshot);
+                    continue;
+                }
+
+                snapshot.Baseline = valueCache.CurrentVersion(serverHandle, snapshot.ItemHandle);
+                Advise(serverHandle, snapshot.ItemHandle);
+                snapshot.Advised = true;
+                advisedByHandle[snapshot.ItemHandle] = snapshot;
+                waiting.Add(snapshot);
+            }
+            catch (Exception exception)
+            {
+                results[snapshot.Index] = FailedRead(serverHandle, snapshot.TagAddress, snapshot.ItemHandle, wasCached: false, exception.Message);
+            }
         }
 
-        // 2. Snapshot lifecycle. Reserve our own item handle, advise, pump
-        //    until we see a fresh OnDataChange (or the deadline elapses),
-        //    then tear it down.
-        int itemHandle = 0;
-        bool advised = false;
+        if (waiting.Count > 0)
+        {
+            WaitForSnapshots(serverHandle, waiting, results, timeout, pumpStep);
+        }
+
+        foreach (SnapshotRead duplicate in sharing)
+        {
+            BulkReadResult? ownerResult = results[duplicate.SharedWith!.Index];
+            if (ownerResult is null)
+            {
+                continue;
+            }
+
+            BulkReadResult fanned = ownerResult.Clone();
+            fanned.TagAddress = duplicate.TagAddress;
+            results[duplicate.Index] = fanned;
+        }
+    }
+
+    private void WaitForSnapshots(
+        int serverHandle,
+        List<SnapshotRead> waiting,
+        BulkReadResult?[] results,
+        TimeSpan timeout,
+        Action pumpStep)
+    {
+        // Phase 3: one shared deadline. Pump until every advised handle has
+        // seen its first OnDataChange since its baseline, or the deadline
+        // elapses.
+        int[] itemHandles = new int[waiting.Count];
+        ulong[] baselines = new ulong[waiting.Count];
+        for (int index = 0; index < waiting.Count; index++)
+        {
+            itemHandles[index] = waiting[index].ItemHandle;
+            baselines[index] = waiting[index].Baseline;
+        }
+
+        bool[] received = new bool[waiting.Count];
+        MxAccessValueCache.CachedValue[] values = new MxAccessValueCache.CachedValue[waiting.Count];
         try
         {
-            itemHandle = AddItem(serverHandle, tagAddress);
-            ulong baseline = valueCache.CurrentVersion(serverHandle, itemHandle);
-            Advise(serverHandle, itemHandle);
-            advised = true;
-
             DateTime deadline = DateTime.UtcNow + timeout;
-            bool gotValue = valueCache.TryWaitForUpdate(
-                serverHandle,
-                itemHandle,
-                baseline,
-                deadline,
-                pumpStep,
-                out MxAccessValueCache.CachedValue snapshot);
-
-            return gotValue
-                ? SucceededRead(serverHandle, tagAddress, itemHandle, wasCached: false, snapshot)
-                : FailedRead(
-                    serverHandle,
-                    tagAddress,
-                    itemHandle,
-                    wasCached: false,
-                    $"ReadBulk timed out after {timeout.TotalMilliseconds:F0} ms waiting for first OnDataChange.");
+            valueCache.TryWaitForUpdates(serverHandle, itemHandles, baselines, deadline, pumpStep, received, values);
         }
         catch (Exception exception)
         {
-            return FailedRead(serverHandle, tagAddress, itemHandle, wasCached: false, exception.Message);
-        }
-        finally
-        {
-            // Snapshot teardown — best-effort. Errors here are noted on the
-            // diagnostic message of the original result (above) by appending
-            // a cleanup suffix; we never re-throw from finally.
-            if (advised)
+            // A pump failure ends the wait for every tag still outstanding;
+            // tags captured before it keep their values.
+            for (int index = 0; index < waiting.Count; index++)
             {
-                try { UnAdvise(serverHandle, itemHandle); } catch { /* swallow — best effort */ }
+                if (!received[index])
+                {
+                    SnapshotRead failed = waiting[index];
+                    results[failed.Index] = FailedRead(serverHandle, failed.TagAddress, failed.ItemHandle, wasCached: false, exception.Message);
+                }
+            }
+        }
+
+        for (int index = 0; index < waiting.Count; index++)
+        {
+            SnapshotRead snapshot = waiting[index];
+            if (results[snapshot.Index] is not null)
+            {
+                continue;
             }
 
-            if (itemHandle != 0)
+            results[snapshot.Index] = received[index]
+                ? SucceededRead(serverHandle, snapshot.TagAddress, snapshot.ItemHandle, wasCached: false, values[index])
+                : FailedRead(
+                    serverHandle,
+                    snapshot.TagAddress,
+                    snapshot.ItemHandle,
+                    wasCached: false,
+                    $"ReadBulk timed out after {timeout.TotalMilliseconds:F0} ms waiting for first OnDataChange.");
+        }
+    }
+
+    private void TearDownSnapshots(
+        int serverHandle,
+        List<SnapshotRead> snapshots,
+        Action pumpStep)
+    {
+        // Phase 4: snapshot teardown — best-effort, never re-throws. Each
+        // distinct handle is unadvised and removed exactly once, even when
+        // MXAccess handed the same id to several tags in the batch.
+        HashSet<int> unadvised = new();
+        HashSet<int> removed = new();
+        int calls = 0;
+        foreach (SnapshotRead snapshot in snapshots)
+        {
+            if (snapshot.Advised && unadvised.Add(snapshot.ItemHandle))
             {
-                try { RemoveItem(serverHandle, itemHandle); } catch { /* swallow — best effort */ }
+                PumpEvery(ref calls, pumpStep);
+                try { UnAdvise(serverHandle, snapshot.ItemHandle); } catch { /* swallow — best effort */ }
             }
         }
+
+        foreach (SnapshotRead snapshot in snapshots)
+        {
+            if (snapshot.ItemHandle != 0 && removed.Add(snapshot.ItemHandle))
+            {
+                PumpEvery(ref calls, pumpStep);
+                try { RemoveItem(serverHandle, snapshot.ItemHandle); } catch { /* swallow — best effort */ }
+            }
+        }
+    }
+
+    private static void PumpEvery(ref int calls, Action pumpStep)
+    {
+        if (calls > 0 && calls % ReadBulkPumpInterval == 0)
+        {
+            try { pumpStep(); } catch { /* swallow — teardown is best effort */ }
+        }
+
+        calls++;
     }
 
     private bool TryGetCachedReadFor(
@@ -1308,6 +1495,41 @@ public sealed class MxAccessSession : IDisposable
         }
     }
 
+    /// <summary>
+    ///     Mutable per-tag state for one ReadBulk snapshot lifecycle. A plain
+    ///     class with get/set properties (no records or init accessors) so it
+    ///     compiles under the worker's net48 target.
+    /// </summary>
+    private sealed class SnapshotRead
+    {
+        public SnapshotRead(int index, string tagAddress)
+        {
+            Index = index;
+            TagAddress = tagAddress;
+        }
+
+        /// <summary>Gets the position of the tag in the request.</summary>
+        public int Index { get; }
+
+        /// <summary>Gets the requested tag address.</summary>
+        public string TagAddress { get; }
+
+        /// <summary>Gets or sets the snapshot item handle, 0 until AddItem succeeds.</summary>
+        public int ItemHandle { get; set; }
+
+        /// <summary>Gets or sets the value-cache version captured before Advise.</summary>
+        public ulong Baseline { get; set; }
+
+        /// <summary>Gets or sets a value indicating whether Advise succeeded.</summary>
+        public bool Advised { get; set; }
+
+        /// <summary>
+        ///     Gets or sets the earlier snapshot in the same batch whose
+        ///     subscription this one shares (same item handle), or null.
+        /// </summary>
+        public SnapshotRead? SharedWith { get; set; }
+    }
+
     private void ThrowIfDisposed()
     {
         if (disposed)

src/ZB.MOM.WW.MxGateway.Worker/MxAccess/MxAccessStaSession.cs

@@ -42,13 +42,24 @@ public sealed class MxAccessStaSession : IWorkerRuntimeSession
     /// <summary>Default alarm poll cadence when the environment says nothing usable.</summary>
     internal static readonly TimeSpan DefaultAlarmPollInterval = TimeSpan.FromMilliseconds(500);
 
-    private readonly TimeSpan alarmPollInterval = ResolveAlarmPollInterval();
+    /// <summary>
+    ///     Consecutive alarm-poll failures that fault the session. Matches the
+    ///     default of <see cref="FailoverSettings.Threshold"/>
+    ///     (<c>AlarmFailoverConfig.consecutive_failure_threshold</c>, default 3):
+    ///     one transient COM hiccup in <c>PollOnce</c> is logged and retried on
+    ///     the next tick, and only a run of failures with no success in between
+    ///     is treated as a dead subscription.
+    /// </summary>
+    internal const int DefaultAlarmPollFailureThreshold = 3;
 
     private readonly IMxAccessComObjectFactory factory;
     private readonly IMxAccessEventSink eventSink;
     private readonly MxAccessEventQueue eventQueue;
     private readonly StaRuntime staRuntime;
-    private readonly Func<MxAccessEventQueue, Action, IMxAccessComObjectFactory, IAlarmCommandHandler>? alarmCommandHandlerFactory;
+    // Pump-aware form: the fourth argument is the STA runtime's pump step, so
+    // handler work that pumps while it waits (the alarm ack writer's
+    // OnWriteComplete wait) also refreshes the STA watchdog heartbeat.
+    private readonly Func<MxAccessEventQueue, Action, IMxAccessComObjectFactory, Action, IAlarmCommandHandler>? alarmCommandHandlerFactory;
     private StaCommandDispatcher? commandDispatcher;
     private MxAccessSession? session;
     private IAlarmCommandHandler? alarmCommandHandler;
@@ -96,6 +107,23 @@ public sealed class MxAccessStaSession : IWorkerRuntimeSession
     {
     }
 
+    /// <summary>
+    /// Initializes a new instance of <see cref="MxAccessStaSession"/> with default dependencies and a
+    /// pump-aware alarm-command handler factory. Its fourth argument is a pump step bound to this
+    /// session's <see cref="StaRuntime"/> (<see cref="StaRuntime.PumpPendingMessages"/>), which
+    /// refreshes the STA watchdog heartbeat on every pump.
+    /// </summary>
+    /// <param name="pumpAwareAlarmCommandHandlerFactory">Factory that constructs the alarm-command handler.</param>
+    internal MxAccessStaSession(Func<MxAccessEventQueue, Action, IMxAccessComObjectFactory, Action, IAlarmCommandHandler> pumpAwareAlarmCommandHandlerFactory)
+    {
+        staRuntime = new StaRuntime();
+        factory = new MxAccessComObjectFactory();
+        eventQueue = new MxAccessEventQueue(MxAccessEventQueue.ResolveCapacity());
+        eventSink = new MxAccessBaseEventSink(eventQueue);
+        alarmCommandHandlerFactory = pumpAwareAlarmCommandHandlerFactory
+            ?? throw new ArgumentNullException(nameof(pumpAwareAlarmCommandHandlerFactory));
+    }
+
     /// <summary>
     /// Initializes a new instance of <see cref="MxAccessStaSession"/> with custom STA runtime and factory.
     /// </summary>
@@ -183,7 +211,9 @@ public sealed class MxAccessStaSession : IWorkerRuntimeSession
         this.factory = factory ?? throw new ArgumentNullException(nameof(factory));
         this.eventSink = eventSink ?? throw new ArgumentNullException(nameof(eventSink));
         this.eventQueue = eventQueue ?? throw new ArgumentNullException(nameof(eventQueue));
-        this.alarmCommandHandlerFactory = alarmCommandHandlerFactory;
+        this.alarmCommandHandlerFactory = alarmCommandHandlerFactory is null
+            ? null
+            : (eq, affinity, comFactory, _) => alarmCommandHandlerFactory(eq, affinity, comFactory);
     }
 
     /// <summary>
@@ -199,6 +229,28 @@ public sealed class MxAccessStaSession : IWorkerRuntimeSession
     /// </summary>
     internal TimeSpan WriteCompletionTimeout { get; set; } = ResolveWriteCompletionTimeout();
 
+    /// <summary>
+    ///     Cadence the alarm poll loop waits between polls. Resolved from the
+    ///     launcher environment; internal-settable as a test seam (set before
+    ///     <see cref="StartAsync(string, int, CancellationToken)"/>).
+    /// </summary>
+    internal TimeSpan AlarmPollInterval { get; set; } = ResolveAlarmPollInterval();
+
+    /// <summary>
+    ///     Consecutive alarm-poll failures that fault the session; clamped to a
+    ///     minimum of 1. Internal-settable as a test seam.
+    /// </summary>
+    internal int AlarmPollFailureThreshold { get; set; } = DefaultAlarmPollFailureThreshold;
+
+    /// <summary>
+    ///     Sink for the transient alarm-poll failure warning. Defaults to
+    ///     <see cref="Console.Error"/>, the stream <c>WorkerConsoleLogger</c>
+    ///     writes to. The line carries the exception type, HRESULT, and
+    ///     failure counts only — never the exception message, which can echo
+    ///     alarm or tag names. Tests substitute a collector.
+    /// </summary>
+    internal Action<string>? AlarmPollWarningSink { get; set; }
+
     /// <summary>
     ///     Resolves the write-completion wait from the launcher-provided
     ///     environment variable; a missing or invalid value falls back to
@@ -266,6 +318,13 @@ public sealed class MxAccessStaSession : IWorkerRuntimeSession
     {
         staRuntime.Start();
 
+        // WRK-02: a post-start STA loop death has no other observer. Record it
+        // on the event queue so the IPC drain loop sends a WorkerFault to the
+        // gateway and ends the pipe session (and with it the worker process),
+        // instead of leaving a live process whose commands queue forever.
+        staRuntime.RegisterLoopFaultHandler(
+            exception => eventQueue.RecordFault(CreateStaLoopFault(exception)));
+
         WorkerReady ready = await staRuntime.InvokeAsync(
             () =>
             {
@@ -288,7 +347,8 @@ public sealed class MxAccessStaSession : IWorkerRuntimeSession
                     alarmCommandHandler = alarmCommandHandlerFactory(
                         eventQueue,
                         EnsureOnAlarmConsumerThread,
-                        factory);
+                        factory,
+                        () => staRuntime.PumpPendingMessages());
                 }
                 commandDispatcher = new StaCommandDispatcher(
                     staRuntime,
@@ -322,13 +382,16 @@ public sealed class MxAccessStaSession : IWorkerRuntimeSession
         IAlarmCommandHandler handler,
         CancellationToken cancellationToken)
     {
+        TimeSpan pollInterval = AlarmPollInterval;
+        int failureThreshold = AlarmPollFailureThreshold < 1 ? 1 : AlarmPollFailureThreshold;
         return Task.Run(async () =>
         {
+            int consecutiveFailures = 0;
             while (!cancellationToken.IsCancellationRequested)
             {
                 try
                 {
-                    await Task.Delay(alarmPollInterval, cancellationToken).ConfigureAwait(false);
+                    await Task.Delay(pollInterval, cancellationToken).ConfigureAwait(false);
                 }
                 catch (OperationCanceledException)
                 {
@@ -361,6 +424,7 @@ public sealed class MxAccessStaSession : IWorkerRuntimeSession
                             }
                         },
                         cancellationToken).ConfigureAwait(false);
+                    consecutiveFailures = 0;
                 }
                 catch (OperationCanceledException)
                 {
@@ -390,19 +454,66 @@ public sealed class MxAccessStaSession : IWorkerRuntimeSession
                     // A real alarm-poll failure (COMException from
                     // GetXmlCurrentAlarms2, malformed-XML parse failure, an
                     // STA-affinity InvalidOperationException from
-                    // EnsureOnAlarmConsumerThread, etc.). Record it as a
-                    // fault on the event queue so a broken alarm subscription
-                    // — or an affinity-invariant regression — becomes
+                    // EnsureOnAlarmConsumerThread, etc.). A single failure is
+                    // logged and retried on the next tick: one transient COM
+                    // hiccup must not tear down the whole session (every data
+                    // subscription with it). Only a run of
+                    // AlarmPollFailureThreshold failures with no success in
+                    // between is recorded as a fault on the event queue, so a
+                    // broken alarm subscription — or an affinity-invariant
+                    // regression, which fails every tick — still becomes
                     // observable on the IPC fault path instead of silently
                     // faulting this never-awaited task. The loop then stops —
                     // the subscription is dead.
-                    eventQueue.RecordFault(CreateAlarmPollFault(exception));
-                    return;
+                    if (staRuntime.LoopFault is not null)
+                    {
+                        // The STA loop itself died; its handler already recorded
+                        // the StaHung fault. This failure is just the fast-fail
+                        // InvokeAsync returns afterwards — recording it would add
+                        // a second, mislabelled alarm-poll fault.
+                        return;
+                    }
+
+                    consecutiveFailures++;
+                    if (consecutiveFailures >= failureThreshold)
+                    {
+                        eventQueue.RecordFault(CreateAlarmPollFault(exception));
+                        return;
+                    }
+
+                    WriteAlarmPollWarning(exception, consecutiveFailures, failureThreshold);
                 }
             }
         }, CancellationToken.None);
     }
 
+    private void WriteAlarmPollWarning(Exception exception, int consecutiveFailures, int failureThreshold)
+    {
+        // Same "level=<L> event=<E> k=v" line shape as WorkerConsoleLogger and
+        // the wnwrap truncation warning. Type and HRESULT only — the exception
+        // message is deliberately left out (it can carry alarm or tag names).
+        string hresult = exception is System.Runtime.InteropServices.COMException comException
+            ? "0x" + comException.HResult.ToString("X8", System.Globalization.CultureInfo.InvariantCulture)
+            : "none";
+        string message = string.Format(
+            System.Globalization.CultureInfo.InvariantCulture,
+            "level=Warning event=AlarmPollFailed exceptionType={0} hresult={1} consecutiveFailures={2} faultThreshold={3}",
+            exception.GetType().FullName,
+            hresult,
+            consecutiveFailures,
+            failureThreshold);
+
+        try
+        {
+            Action<string> sink = AlarmPollWarningSink ?? Console.Error.WriteLine;
+            sink(message);
+        }
+        catch
+        {
+            // Logging must never end the poll loop.
+        }
+    }
+
     private void EnsureOnAlarmConsumerThread()
     {
         AssertOnAlarmConsumerThread(alarmConsumerThreadId, Environment.CurrentManagedThreadId);
@@ -435,6 +546,41 @@ public sealed class MxAccessStaSession : IWorkerRuntimeSession
         }
     }
 
+    /// <summary>
+    ///     Builds the fault reported when the STA loop dies after start. Uses
+    ///     <see cref="WorkerFaultCategory.StaHung"/>: the gateway already treats
+    ///     that category as "this worker's apartment can no longer run COM
+    ///     work", which is exactly the state a dead loop leaves behind.
+    /// </summary>
+    internal static WorkerFault CreateStaLoopFault(Exception exception)
+    {
+        // Type and HRESULT only, never exception.Message — the same type-only
+        // rule every other worker failure line follows.
+        string message = string.Format(
+            System.Globalization.CultureInfo.InvariantCulture,
+            "Worker STA loop terminated unexpectedly: exceptionType={0} hresult=0x{1:X8}",
+            exception.GetType().FullName,
+            exception.HResult);
+        WorkerFault fault = new()
+        {
+            Category = WorkerFaultCategory.StaHung,
+            ExceptionType = exception.GetType().FullName ?? string.Empty,
+            DiagnosticMessage = message,
+            ProtocolStatus = new ProtocolStatus
+            {
+                Code = ProtocolStatusCode.WorkerUnavailable,
+                Message = message,
+            },
+        };
+
+        if (exception is System.Runtime.InteropServices.COMException comException)
+        {
+            fault.Hresult = comException.HResult;
+        }
+
+        return fault;
+    }
+
     private static WorkerFault CreateAlarmPollFault(Exception exception)
     {
         string message =

src/ZB.MOM.WW.MxGateway.Worker/MxAccess/MxAccessValueCache.cs

@@ -162,6 +162,46 @@ public sealed class MxAccessValueCache
         }
     }
 
+    /// <summary>
+    ///     Removes every cache slot recorded under a server handle. The session
+    ///     calls this when a server handle is unregistered: MXAccess drops all of
+    ///     that server's items with it, so the per-item <see cref="Remove"/>
+    ///     never runs for them and their entries would otherwise outlive the
+    ///     handle (and could be served to a later server that reuses the id).
+    /// </summary>
+    /// <param name="serverHandle">MXAccess server handle that was unregistered.</param>
+    public void RemoveServer(int serverHandle)
+    {
+        lock (syncRoot)
+        {
+            List<long> stale = new();
+            foreach (long key in entries.Keys)
+            {
+                if ((int)(key >> 32) == serverHandle)
+                {
+                    stale.Add(key);
+                }
+            }
+
+            foreach (long key in stale)
+            {
+                entries.Remove(key);
+            }
+        }
+    }
+
+    /// <summary>Gets the number of handle pairs with a cached value (test seam).</summary>
+    internal int Count
+    {
+        get
+        {
+            lock (syncRoot)
+            {
+                return entries.Count;
+            }
+        }
+    }
+
     /// <summary>
     ///     Waits until the cache entry's version exceeds <paramref name="sinceVersion"/>
     ///     or the deadline elapses, calling <paramref name="pumpStep"/> on every
@@ -231,6 +271,117 @@ public sealed class MxAccessValueCache
         }
     }
 
+    /// <summary>
+    ///     Multi-handle form of <see cref="TryWaitForUpdate"/>: waits until every
+    ///     handle in <paramref name="itemHandles"/> has an entry newer than its
+    ///     matching <paramref name="sinceVersions"/> slot, or the single shared
+    ///     deadline elapses. Each handle is captured the first time an update is
+    ///     observed for it (its first OnDataChange since the baseline), exactly
+    ///     as a dedicated per-handle wait would have captured it, so a bulk
+    ///     snapshot read waits at most one timeout in total rather than one per
+    ///     tag.
+    /// </summary>
+    /// <remarks>
+    ///     Pumps unconditionally at the top of every iteration and blocks the
+    ///     idle part of each iteration in
+    ///     <see cref="StaWaitHelper.WaitForSignalOrMessages"/> — the same pump
+    ///     discipline as <see cref="TryWaitForUpdate"/>, for the same reason.
+    /// </remarks>
+    /// <param name="serverHandle">MXAccess server handle shared by every item.</param>
+    /// <param name="itemHandles">Item handles to wait on.</param>
+    /// <param name="sinceVersions">Per-handle version baselines captured before advising.</param>
+    /// <param name="deadlineUtc">Absolute UTC deadline shared by every handle.</param>
+    /// <param name="pumpStep">Action that pumps any pending Windows messages.</param>
+    /// <param name="received">Output: <see langword="true"/> at index i when handle i received an update before the deadline.</param>
+    /// <param name="values">Output: the captured value at index i when <paramref name="received"/>[i] is set.</param>
+    /// <param name="pollIntervalMs">Upper bound on one idle wait slice.</param>
+    /// <returns>The number of handles that received an update before the deadline.</returns>
+    public int TryWaitForUpdates(
+        int serverHandle,
+        IReadOnlyList<int> itemHandles,
+        IReadOnlyList<ulong> sinceVersions,
+        DateTime deadlineUtc,
+        Action pumpStep,
+        bool[] received,
+        CachedValue[] values,
+        int pollIntervalMs = 5)
+    {
+        if (itemHandles is null)
+        {
+            throw new ArgumentNullException(nameof(itemHandles));
+        }
+
+        if (sinceVersions is null)
+        {
+            throw new ArgumentNullException(nameof(sinceVersions));
+        }
+
+        if (pumpStep is null)
+        {
+            throw new ArgumentNullException(nameof(pumpStep));
+        }
+
+        if (received is null)
+        {
+            throw new ArgumentNullException(nameof(received));
+        }
+
+        if (values is null)
+        {
+            throw new ArgumentNullException(nameof(values));
+        }
+
+        int count = itemHandles.Count;
+        if (sinceVersions.Count != count || received.Length < count || values.Length < count)
+        {
+            throw new ArgumentException("Handle, baseline, and output arrays must have matching lengths.");
+        }
+
+        int receivedCount = 0;
+        for (int index = 0; index < count; index++)
+        {
+            if (received[index])
+            {
+                receivedCount++;
+            }
+        }
+
+        while (true)
+        {
+            pumpStep();
+
+            for (int index = 0; index < count; index++)
+            {
+                if (received[index])
+                {
+                    continue;
+                }
+
+                if (TryGet(serverHandle, itemHandles[index], out CachedValue value) && value.Version > sinceVersions[index])
+                {
+                    received[index] = true;
+                    values[index] = value;
+                    receivedCount++;
+                }
+            }
+
+            if (receivedCount >= count)
+            {
+                return receivedCount;
+            }
+
+            TimeSpan remaining = deadlineUtc - DateTime.UtcNow;
+            if (remaining <= TimeSpan.Zero)
+            {
+                return receivedCount;
+            }
+
+            StaWaitHelper.WaitForSignalOrMessages(
+                updateSignal,
+                StaWaitHelper.ClampWaitMilliseconds(remaining, pollIntervalMs));
+        }
+    }
+
     /// <summary>Returns the current version for a handle pair, or 0 if no entry exists.</summary>
     /// <param name="serverHandle">MXAccess server handle.</param>
     /// <param name="itemHandle">MXAccess item handle.</param>

src/ZB.MOM.WW.MxGateway.Worker/MxAccess/MxAccessWriteCompletionCache.cs

@@ -184,6 +184,57 @@ public sealed class MxAccessWriteCompletionCache
         }
     }
 
+    /// <summary>
+    ///     Forgets the completion entry of a handle pair whose MXAccess item was
+    ///     removed, so a long-lived writer that drops and re-adds items does not
+    ///     keep one entry per dropped handle for the life of the worker.
+    /// </summary>
+    /// <param name="serverHandle">MXAccess server handle.</param>
+    /// <param name="itemHandle">MXAccess item handle that was removed.</param>
+    public void Remove(
+        int serverHandle,
+        int itemHandle)
+    {
+        lock (syncRoot)
+        {
+            entries.Remove(CreateItemKey(serverHandle, itemHandle));
+        }
+    }
+
+    /// <summary>Forgets every completion entry recorded under a server handle that was unregistered.</summary>
+    /// <param name="serverHandle">MXAccess server handle that was unregistered.</param>
+    public void RemoveServer(int serverHandle)
+    {
+        lock (syncRoot)
+        {
+            List<long> stale = new();
+            foreach (long key in entries.Keys)
+            {
+                if ((int)(key >> 32) == serverHandle)
+                {
+                    stale.Add(key);
+                }
+            }
+
+            foreach (long key in stale)
+            {
+                entries.Remove(key);
+            }
+        }
+    }
+
+    /// <summary>Gets the number of handle pairs with a recorded completion (test seam).</summary>
+    internal int Count
+    {
+        get
+        {
+            lock (syncRoot)
+            {
+                return entries.Count;
+            }
+        }
+    }
+
     private static long CreateItemKey(
         int serverHandle,
         int itemHandle)

src/ZB.MOM.WW.MxGateway.Worker/Sta/IStaMessagePump.cs

@@ -0,0 +1,22 @@
+using System;
+using System.Threading;
+
+namespace ZB.MOM.WW.MxGateway.Worker.Sta;
+
+/// <summary>
+///     The two Win32 pump operations <see cref="StaRuntime"/> drives on its
+///     STA thread. <see cref="StaMessagePump"/> is the only production
+///     implementation; the seam exists so Worker.Tests can observe pump
+///     interleaving and inject a pump failure without a real message queue.
+/// </summary>
+internal interface IStaMessagePump
+{
+    /// <summary>Waits for a command wake event or Windows messages.</summary>
+    /// <param name="commandWakeEvent">Event signalled when command work is available.</param>
+    /// <param name="timeout">Maximum time to wait.</param>
+    void WaitForWorkOrMessages(WaitHandle commandWakeEvent, TimeSpan timeout);
+
+    /// <summary>Pumps and dispatches all pending Windows messages.</summary>
+    /// <returns>The number of messages dispatched.</returns>
+    int PumpPendingMessages();
+}

src/ZB.MOM.WW.MxGateway.Worker/Sta/StaMessagePump.cs

@@ -6,7 +6,7 @@ using Microsoft.Win32.SafeHandles;
 namespace ZB.MOM.WW.MxGateway.Worker.Sta;
 
 /// <summary>Pumps Windows messages on the STA thread to allow MXAccess COM events to deliver.</summary>
-public sealed class StaMessagePump
+public sealed class StaMessagePump : IStaMessagePump
 {
     private const uint Infinite = 0xFFFFFFFF;
     private const uint PmRemove = 0x0001;

src/ZB.MOM.WW.MxGateway.Worker/Sta/StaRuntime.cs

@@ -8,7 +8,7 @@ namespace ZB.MOM.WW.MxGateway.Worker.Sta;
 public sealed class StaRuntime : IDisposable
 {
     private readonly IStaComApartmentInitializer comApartmentInitializer;
-    private readonly StaMessagePump messagePump;
+    private readonly IStaMessagePump messagePump;
     private readonly ConcurrentQueue<IStaWorkItem> commandQueue = new();
     private readonly AutoResetEvent commandWakeEvent = new(false);
     private readonly ManualResetEventSlim startedEvent = new(false);
@@ -20,6 +20,8 @@ public sealed class StaRuntime : IDisposable
     private bool startRequested;
     private bool shutdownRequested;
     private Exception? startupException;
+    private Exception? loopFault;
+    private Action<Exception>? loopFaultHandler;
     private long lastActivityUtcTicks;
     private bool comInitialized;
 
@@ -41,6 +43,21 @@ public sealed class StaRuntime : IDisposable
         IStaComApartmentInitializer comApartmentInitializer,
         StaMessagePump messagePump,
         TimeSpan idlePumpInterval)
+        : this(comApartmentInitializer, (IStaMessagePump)messagePump, idlePumpInterval)
+    {
+    }
+
+    /// <summary>
+    /// Initializes a new instance of <see cref="StaRuntime"/> over any
+    /// <see cref="IStaMessagePump"/> (test seam).
+    /// </summary>
+    /// <param name="comApartmentInitializer">COM apartment initializer.</param>
+    /// <param name="messagePump">Message pump for the STA thread.</param>
+    /// <param name="idlePumpInterval">Interval for idle message pump waits.</param>
+    internal StaRuntime(
+        IStaComApartmentInitializer comApartmentInitializer,
+        IStaMessagePump messagePump,
+        TimeSpan idlePumpInterval)
     {
         this.comApartmentInitializer = comApartmentInitializer
             ?? throw new ArgumentNullException(nameof(comApartmentInitializer));
@@ -79,6 +96,59 @@ public sealed class StaRuntime : IDisposable
     /// </summary>
     public bool IsRunning => startedEvent.IsSet && !stoppedEvent.IsSet;
 
+    /// <summary>
+    /// Gets the exception that terminated the STA loop after a successful
+    /// <see cref="Start"/>, or <see langword="null"/> while the loop is healthy
+    /// (or stopped by a requested shutdown). Startup failures are not reported
+    /// here — <see cref="Start"/> throws them.
+    /// </summary>
+    public Exception? LoopFault
+    {
+        get
+        {
+            lock (gate)
+            {
+                return loopFault;
+            }
+        }
+    }
+
+    /// <summary>
+    /// Registers the callback invoked once, on the dying STA thread, when the
+    /// STA loop terminates with an exception after a successful
+    /// <see cref="Start"/>. When the loop has already died the callback runs
+    /// immediately on the calling thread, so a fault that lands between
+    /// <see cref="Start"/> and registration is never lost. Only one handler is
+    /// kept; a later registration replaces an earlier one.
+    /// </summary>
+    /// <remarks>
+    /// Without this, a post-start loop death (a failed
+    /// <c>MsgWaitForMultipleObjectsEx</c>, or an exception escaping a dispatched
+    /// window message) left the worker process alive with no STA: every later
+    /// command queued forever and the gateway only learned of it through the
+    /// heartbeat watchdog, if at all.
+    /// </remarks>
+    /// <param name="handler">Callback receiving the terminating exception.</param>
+    public void RegisterLoopFaultHandler(Action<Exception> handler)
+    {
+        if (handler is null)
+        {
+            throw new ArgumentNullException(nameof(handler));
+        }
+
+        Exception? alreadyFaulted;
+        lock (gate)
+        {
+            loopFaultHandler = handler;
+            alreadyFaulted = shutdownRequested ? null : loopFault;
+        }
+
+        if (alreadyFaulted is not null)
+        {
+            handler(alreadyFaulted);
+        }
+    }
+
     /// <summary>
     ///     Pumps any pending Windows messages on the calling thread and refreshes
     ///     the STA activity timestamp. Intended for commands that synchronously
@@ -176,6 +246,14 @@ public sealed class StaRuntime : IDisposable
 
         lock (gate)
         {
+            if (loopFault is not null)
+            {
+                // The loop is gone; nothing would ever dequeue this item.
+                return Task.FromException<T>(new InvalidOperationException(
+                    "The worker STA loop terminated unexpectedly.",
+                    loopFault));
+            }
+
             if (shutdownRequested)
             {
                 return Task.FromException<T>(new StaRuntimeShutdownException());
@@ -244,12 +322,14 @@ public sealed class StaRuntime : IDisposable
 
     private void ThreadMain()
     {
+        bool started = false;
         try
         {
             StaThreadId = Thread.CurrentThread.ManagedThreadId;
             comApartmentInitializer.Initialize();
             comInitialized = true;
             MarkActivity();
+            started = true;
             startedEvent.Set();
 
             while (!IsShutdownRequested())
@@ -264,8 +344,44 @@ public sealed class StaRuntime : IDisposable
         }
         catch (Exception exception)
         {
-            startupException = exception;
-            startedEvent.Set();
+            if (!started)
+            {
+                // Start() is blocked on startedEvent and rethrows this.
+                startupException = exception;
+                startedEvent.Set();
+            }
+            else
+            {
+                // Post-start loop death. Start() returned long ago, so nothing
+                // would ever read startupException: record the fault where
+                // InvokeAsync and the registered handler see it. Set under the
+                // gate so every item enqueued before it is still in the queue
+                // for the finally block's CancelQueuedCommands, and every
+                // InvokeAsync after it fails fast instead of queueing forever.
+                Action<Exception>? handler;
+                lock (gate)
+                {
+                    loopFault = exception;
+
+                    // A loop that throws while a requested shutdown is already
+                    // in progress is being torn down on purpose; reporting it
+                    // as a worker fault would turn a clean stop into a fault.
+                    handler = shutdownRequested ? null : loopFaultHandler;
+                }
+
+                if (handler is not null)
+                {
+                    try
+                    {
+                        handler(exception);
+                    }
+                    catch
+                    {
+                        // The handler reports the fault; a failure inside it
+                        // must not skip the apartment cleanup below.
+                    }
+                }
+            }
         }
         finally
         {
@@ -292,6 +408,17 @@ public sealed class StaRuntime : IDisposable
             MarkActivity();
             workItem.Execute();
             MarkActivity();
+
+            // Pump between work items, not only once the whole queue is
+            // drained: a burst of queued commands (bulk callers, the alarm
+            // poll, a reconnecting client replaying subscriptions) would
+            // otherwise hold every inbound COM callback — OnDataChange,
+            // OnWriteComplete — behind the entire backlog. PeekMessage with an
+            // empty queue is cheap, so the per-item cost is negligible.
+            if (!commandQueue.IsEmpty)
+            {
+                messagePump.PumpPendingMessages();
+            }
         }
     }
 

src/ZB.MOM.WW.MxGateway.Worker/WorkerApplication.cs

@@ -14,10 +14,61 @@ public static class WorkerApplication
     /// <returns>The process exit code.</returns>
     public static int Run(string[] args)
     {
+        WorkerConsoleLogger logger = new(Console.Error);
+
+        // WRK-09: installed only on the production entry point (not the
+        // injectable overloads tests call) because the hook is process-global.
+        AppDomain.CurrentDomain.UnhandledException += (_, eventArgs) =>
+            HandleUnhandledException(
+                logger,
+                eventArgs.ExceptionObject,
+                eventArgs.IsTerminating,
+                Environment.Exit);
+
         return Run(
             args,
             new EnvironmentVariableWorkerEnvironment(),
-            new WorkerConsoleLogger(Console.Error));
+            logger);
+    }
+
+    /// <summary>
+    ///     Body of the worker's <c>AppDomain.CurrentDomain.UnhandledException</c>
+    ///     hook. Logs one <c>WorkerUnhandledException</c> line and exits the
+    ///     process with <see cref="WorkerExitCode.UnhandledException"/>, so the
+    ///     gateway's launcher sees a defined worker exit code rather than the
+    ///     CLR's crash code.
+    /// </summary>
+    /// <remarks>
+    ///     Logs the exception type only — never its message or stack, which
+    ///     can carry tag values or credentials passed through a command — the
+    ///     same rule every other worker failure line follows. A logging
+    ///     failure is swallowed so the exit always happens.
+    /// </remarks>
+    /// <param name="logger">Worker logger.</param>
+    /// <param name="exceptionObject">The unhandled exception object (not necessarily an <see cref="Exception"/>).</param>
+    /// <param name="isTerminating">Whether the CLR is terminating the process.</param>
+    /// <param name="exit">Process-exit action; <see cref="Environment.Exit"/> in production.</param>
+    internal static void HandleUnhandledException(
+        IWorkerLogger logger,
+        object? exceptionObject,
+        bool isTerminating,
+        Action<int> exit)
+    {
+        try
+        {
+            logger.Error("WorkerUnhandledException", new Dictionary<string, object?>
+            {
+                ["exit_code"] = WorkerExitCode.UnhandledException,
+                ["exception_type"] = exceptionObject?.GetType().FullName ?? "null",
+                ["is_terminating"] = isTerminating,
+            });
+        }
+        catch
+        {
+            // Never let a logging failure skip the defined exit.
+        }
+
+        exit((int)WorkerExitCode.UnhandledException);
     }
 
     /// <summary>Initializes and runs the worker with custom environment and logging.</summary>