fix(worker): review follow-up — ghost cache entries, ReadBulk dedupe/pumping, fault hygiene

- AddItem/AddItem2/AddBufferedItem drop stale cache entries for a reused handle id
- ReadBulk: dedupe handles within a batch (advise/teardown once, fan out result);
  pump every 16 items in add/advise and teardown phases
- alarm poll stops without recording a fault once StaRuntime.LoopFault is set
- StaRuntime skips the loop-fault handler during a requested shutdown
- StaHung loop fault diagnostic carries type + HRESULT only
- gateway.md documents ReadBulk's shared deadline

CLAUDE.md

@@ -126,7 +126,7 @@ powershell -ExecutionPolicy Bypass -File scripts/run-client-e2e-tests.ps1
 - **One worker per session** (invariant). Multi-subscriber event fan-out and reconnect-with-replay have shipped and are config-gated: `AllowMultipleEventSubscribers` (default `false`) enables fan-out up to `MaxEventSubscribersPerSession` (default `8`); `DetachGraceSeconds` (default `30`) retains a session after its last subscriber drops so clients can reconnect; `ReplayBufferCapacity` / `ReplayRetentionSeconds` control how much event history the replay ring keeps. Default config is single-subscriber (`AllowMultipleEventSubscribers` off), but detach-grace and replay retention are **on** by default (`DetachGraceSeconds=30`, `ReplayBufferCapacity=1024`, `ReplayRetentionSeconds=300`): a detached session is retained for 30 s and recent events are buffered for reconnect. The reconnect protocol is consumable end-to-end: a resuming `StreamEvents` (via `after_worker_sequence`) that predates the retained ring gets a `ReplayGap` sentinel, and all five official clients surface it as a typed signal. Orphan-worker reattach after a gateway restart is **deferred, not planned** — see `docs/DesignDecisions.md` (Session-Resilience Epic Scope, session-resilience epic Phase 5); the invariant on the next line stands. See `docs/DesignDecisions.md` and `docs/Sessions.md`.
 - **Gateway restart does not reattach orphan workers.** The first version terminates orphaned workers on startup; do not design code paths that assume reattachment.
 - **No Blazor UI component libraries.** Dashboard uses local Bootstrap CSS/JS only — do not introduce MudBlazor, Radzen, FluentUI, etc.
-- **Don't log secrets or full tag values by default.** API keys, passwords, `WriteSecured` payloads, and `AuthenticateUser` credentials must never reach logs. Value logging is opt-in and redacted.
+- **Don't log secrets or full tag values by default.** API keys, passwords, `WriteSecured` payloads, `AuthenticateUser` credentials, and `AcknowledgeAlarm` `operator_password` must never reach logs. Value logging is opt-in and redacted.
 - **Generated code** under `src/ZB.MOM.WW.MxGateway.Contracts/Generated/`, `clients/*/generated*/`, `clients/python/src/zb_mom_ww_mxgateway/generated/`, etc., is build output. Don't hand-edit. To regenerate, build the contracts project (`dotnet build src/ZB.MOM.WW.MxGateway.Contracts/ZB.MOM.WW.MxGateway.Contracts.csproj`) or run the per-client generation step in that client's README.
 - **Documentation style** (`StyleGuide.md`): PascalCase filenames, no marketing language, present tense, explain *why* not *what*.
 - **Update docs in the same change as the source.** When public APIs, contracts, configuration, build steps, security behavior, event shapes, value conversion, status mapping, or lifecycle rules change, the affected docs (`gateway.md`, `docs/`, client READMEs, design docs) must change in the same commit. Don't leave stale prose describing old behavior.

clients/dotnet/README.md

@@ -143,8 +143,10 @@ the same `MxEventStreamItem` surface.
 For alarms, the client exposes `QueryActiveAlarmsAsync` (one-shot snapshot of
 the active alarms the gateway's central monitor currently holds),
 `StreamAlarmsAsync` (server-streaming feed of alarm-state-change messages
-keyed by the same monitor), and `AcknowledgeAlarmAsync` (ack by alarm
+keyed by the same monitor), and `AcknowledgeAlarmAsync` (acknowledges as
-reference, optional comment, ack target). All three accept a cancellation
+the operator: the gateway authenticates `OperatorUser`/`OperatorPassword` and
+writes the comment to the alarm's `AckMsg`, so the Historian records the
+operator). All three accept a cancellation
 token and pass through the `MxGateway:Alarms` configuration on the
 server — when alarms are disabled, the gateway returns an empty list / empty
 stream rather than failing.
@@ -291,10 +293,15 @@ dotnet run --project clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli -- write --se
 dotnet run --project clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli -- write2 --session-id <id> --server-handle 1 --item-handle 1 --type int32 --value 123 --timestamp 2026-01-01T00:00:00Z --json
 dotnet run --project clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli -- stream-events --session-id <id> --max-events 1 --json
 dotnet run --project clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli -- stream-alarms --filter-prefix Area001 --max-events 4 --json
-dotnet run --project clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli -- acknowledge-alarm --reference "\\Galaxy\Area001.Pump001.PumpFault" --comment "ack from cli" --operator operator1 --json
+dotnet run --project clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli -- acknowledge-alarm --reference "\\Galaxy\Area001.Pump001.PumpFault" --comment "ack from cli" --operator operator1 --operator-password-env MXGATEWAY_OPERATOR_PASSWORD --json
 dotnet run --project clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli -- smoke --endpoint http://localhost:5000 --api-key-env MXGATEWAY_API_KEY --item Area001.Pump001.Speed --json
 ```
 
+`acknowledge-alarm` reads the operator password from `--operator-password` or the
+environment variable named by `--operator-password-env` (default
+`MXGATEWAY_OPERATOR_PASSWORD`); the password is never echoed and is redacted from
+any error output. Prefer the environment variable to keep it off the command line.
+
 `smoke` opens a session, registers a client, adds one item, advises it,
 optionally writes a value when `--type` and `--value` are supplied, reads a
 bounded event stream, and closes the session in a `finally` block. CLI error

clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli/MxGatewayClientCli.cs

@@ -185,6 +185,7 @@ public static class MxGatewayClientCli
                 exception.Message,
                 apiKey,
                 TryResolveVerifyUserPassword(arguments),
+                ResolveOperatorPassword(arguments),
                 arguments.GetOptional("value"));
 
             if (forceJsonErrors || arguments.HasFlag("json"))
@@ -352,6 +353,12 @@ public static class MxGatewayClientCli
     /// </summary>
     private const string DefaultVerifyPasswordEnvironmentName = "MXGATEWAY_VERIFY_PASSWORD";
 
+    /// <summary>
+    ///     Default environment variable holding the operator password for
+    ///     <c>acknowledge-alarm</c> when <c>--operator-password-env</c> is not given.
+    /// </summary>
+    private const string OperatorPasswordEnvironmentName = "MXGATEWAY_OPERATOR_PASSWORD";
+
     /// <summary>
     ///     Legacy environment variable, still honoured as a deprecated fallback
     ///     for one release so existing scripts keep working.
@@ -378,6 +385,30 @@ public static class MxGatewayClientCli
             : environmentName;
     }
 
+    /// <summary>
+    ///     Resolves the <c>acknowledge-alarm</c> operator password: <c>--operator-password</c>,
+    ///     then the environment variable named by <c>--operator-password-env</c>
+    ///     (default <c>MXGATEWAY_OPERATOR_PASSWORD</c>). An empty value is treated as
+    ///     absent. The value is never echoed; it is also fed to the error redactor.
+    /// </summary>
+    private static string? ResolveOperatorPassword(CliArguments arguments)
+    {
+        string? password = arguments.GetOptional("operator-password");
+        if (!string.IsNullOrEmpty(password))
+        {
+            return password;
+        }
+
+        string environmentName = arguments.GetOptional("operator-password-env") ?? string.Empty;
+        if (string.IsNullOrEmpty(environmentName))
+        {
+            environmentName = OperatorPasswordEnvironmentName;
+        }
+
+        password = Environment.GetEnvironmentVariable(environmentName);
+        return string.IsNullOrEmpty(password) ? null : password;
+    }
+
     /// <summary>
     ///     Resolves the effective MXAccess verify-user credential in this
     ///     order: <c>--password</c>, the deprecated <c>--verify-user-password</c>
@@ -1535,6 +1566,8 @@ public static class MxGatewayClientCli
             AlarmFullReference = arguments.GetRequired("reference"),
             Comment = arguments.GetOptional("comment") ?? string.Empty,
             OperatorUser = arguments.GetOptional("operator") ?? string.Empty,
+            OperatorDomain = arguments.GetOptional("operator-domain") ?? string.Empty,
+            OperatorPassword = ResolveOperatorPassword(arguments) ?? string.Empty,
         };
 
         return WriteReplyAsync(
@@ -2465,7 +2498,7 @@ public static class MxGatewayClientCli
         writer.WriteLine("mxgw-dotnet bench-read-bulk [--duration-seconds <n>] [--warmup-seconds <n>] [--bulk-size <n>] [--tag-start <n>] [--tag-prefix <s>] [--tag-attribute <s>] [--timeout-ms <n>] [--client-name <name>]");
         writer.WriteLine("mxgw-dotnet stream-events --session-id <id> [--max-events <n>] [--json]");
         writer.WriteLine("mxgw-dotnet stream-alarms [--filter-prefix <ref>] [--max-events <n>] [--json] [--jsonl]");
-        writer.WriteLine("mxgw-dotnet acknowledge-alarm --reference <ref> [--comment <text>] [--operator <user>] [--json]");
+        writer.WriteLine("mxgw-dotnet acknowledge-alarm --reference <ref> --operator <user> [--operator-domain <dom>] [--operator-password <pw> | --operator-password-env <VAR>] [--comment <text>] [--json]");
         writer.WriteLine("mxgw-dotnet write --session-id <id> --server-handle <n> --item-handle <n> --type <type> --value <value> [--json]");
         writer.WriteLine("mxgw-dotnet write2 --session-id <id> --server-handle <n> --item-handle <n> --type <type> --value <value> [--timestamp <iso>] [--json]");
         writer.WriteLine("mxgw-dotnet smoke --item <ref> [--value <value> --type <type>] [--json]");

clients/dotnet/ZB.MOM.WW.MxGateway.Client.Tests/MxGatewayClientAlarmsTests.cs

@@ -36,6 +36,8 @@ public sealed class MxGatewayClientAlarmsTests
             AlarmFullReference = "Tank01.Level.HiHi",
             Comment = "investigating",
             OperatorUser = "alice",
+            OperatorPassword = "pw-secret",
+            OperatorDomain = "CORP",
         });
 
         Assert.Equal(ProtocolStatusCode.Ok, reply.ProtocolStatus.Code);
@@ -45,9 +47,35 @@ public sealed class MxGatewayClientAlarmsTests
         Assert.Equal("Tank01.Level.HiHi", call.Request.AlarmFullReference);
         Assert.Equal("investigating", call.Request.Comment);
         Assert.Equal("alice", call.Request.OperatorUser);
+        Assert.Equal("pw-secret", call.Request.OperatorPassword);
+        Assert.Equal("CORP", call.Request.OperatorDomain);
         Assert.Equal("Bearer test-api-key", call.CallOptions.Headers?.GetValue("authorization"));
     }
 
+    /// <summary>AcknowledgeAlarmAsync redacts the operator password from a failing call's message.</summary>
+    /// <returns>A task that represents the asynchronous operation.</returns>
+    [Fact]
+    public async Task AcknowledgeAlarmAsync_FailureMessage_DoesNotContainOperatorPassword()
+    {
+        FakeGatewayTransport transport = CreateTransport();
+
+        // The real transport maps RpcException to MxGatewayException using the status
+        // detail; the fake passes exceptions verbatim, so enqueue the mapped shape.
+        transport.AcknowledgeAlarmExceptions.Enqueue(new MxGatewayException("boom pw-secret"));
+        await using MxGatewayClient client = CreateClient(transport);
+
+        MxGatewayException ex = await Assert.ThrowsAsync<MxGatewayException>(
+            () => client.AcknowledgeAlarmAsync(new AcknowledgeAlarmRequest
+            {
+                AlarmFullReference = "Tank01.Level.HiHi",
+                OperatorUser = "alice",
+                OperatorPassword = "pw-secret",
+            }));
+
+        Assert.DoesNotContain("pw-secret", ex.Message);
+        Assert.Contains("boom", ex.Message);
+    }
+
     /// <summary>AcknowledgeAlarmAsync honors cancellation.</summary>
     /// <returns>A task that represents the asynchronous operation.</returns>
     [Fact]

clients/dotnet/ZB.MOM.WW.MxGateway.Client.Tests/MxGatewayClientCliTests.cs

@@ -763,6 +763,7 @@ public sealed class MxGatewayClientCliTests
     [Fact]
     public async Task RunAsync_AcknowledgeAlarm_BuildsRequestAndPrintsJsonReply()
     {
+        Environment.SetEnvironmentVariable("MXGW_TEST_OPPW", "op-pw-secret");
         using var output = new StringWriter();
         using var error = new StringWriter();
         FakeCliClient fakeClient = new();
@@ -786,6 +787,10 @@ public sealed class MxGatewayClientCliTests
                 "ack from cli",
                 "--operator",
                 "operator1",
+                "--operator-domain",
+                "CORP",
+                "--operator-password-env",
+                "MXGW_TEST_OPPW",
                 "--json",
             ],
             output,
@@ -797,6 +802,9 @@ public sealed class MxGatewayClientCliTests
         Assert.Equal("Tank01.Level.HiHi", request.AlarmFullReference);
         Assert.Equal("ack from cli", request.Comment);
         Assert.Equal("operator1", request.OperatorUser);
+        Assert.Equal("CORP", request.OperatorDomain);
+        Assert.Equal("op-pw-secret", request.OperatorPassword);
+        Assert.DoesNotContain("op-pw-secret", output.ToString());
         Assert.Contains("ack-fixture", output.ToString());
         Assert.Equal(string.Empty, error.ToString());
     }

clients/dotnet/ZB.MOM.WW.MxGateway.Client/MxGatewayClient.cs

@@ -183,24 +183,32 @@ public sealed class MxGatewayClient : IAsyncDisposable
     }
 
     /// <summary>
-    /// Acknowledges an active MXAccess alarm condition through the gateway. The
+    /// Acknowledges an alarm as the operator. The gateway authenticates
-    /// gateway authenticates the request against the API key's <c>invoke:alarm-ack</c>
+    /// <see cref="AcknowledgeAlarmRequest.OperatorUser"/> with
-    /// scope and forwards the acknowledge to the worker's MXAccess session;
+    /// <see cref="AcknowledgeAlarmRequest.OperatorPassword"/> and writes the
-    /// the resulting <see cref="MxStatusProxy"/> is returned in the reply.
+    /// comment to the alarm's AckMsg attribute, so the alarm history records
+    /// the operator. The password is redacted from any surfaced error.
     /// </summary>
-    /// <param name="request">The acknowledge request — alarm reference, comment, operator user.</param>
+    /// <param name="request">The acknowledge request: alarm reference, comment, operator credentials.</param>
     /// <param name="cancellationToken">Cancellation token for the operation.</param>
     /// <returns>The acknowledge reply with protocol + native MxStatus.</returns>
-    public Task<AcknowledgeAlarmReply> AcknowledgeAlarmAsync(
+    public async Task<AcknowledgeAlarmReply> AcknowledgeAlarmAsync(
         AcknowledgeAlarmRequest request,
         CancellationToken cancellationToken = default)
     {
         ArgumentNullException.ThrowIfNull(request);
         ThrowIfDisposed();
 
-        return ExecuteSafeUnaryAsync(
+        try
-            token => _transport.AcknowledgeAlarmAsync(request, CreateCallOptions(token)),
+        {
-            cancellationToken);
+            return await ExecuteSafeUnaryAsync(
+                token => _transport.AcknowledgeAlarmAsync(request, CreateCallOptions(token)),
+                cancellationToken).ConfigureAwait(false);
+        }
+        catch (MxGatewayException ex) when (request.OperatorPassword.Length > 0)
+        {
+            throw MxGatewaySecretRedaction.Redacted(ex, request.OperatorPassword);
+        }
     }
 
     /// <summary>

clients/go/README.md

@@ -140,7 +140,11 @@ sentinel typed and observable; it never synthesizes or swallows it.
 
 For alarms, the package exposes `Client.QueryActiveAlarms` for one-shot
 snapshots, `Client.StreamAlarms` for the server-streaming feed, and
-`Client.AcknowledgeAlarm` to ack an alarm by full reference. The streaming
+`Client.AcknowledgeAlarm` to ack an alarm by full reference. Acks carry the
+operator's Galaxy credentials (`OperatorUser` required, `OperatorPassword`,
+optional `OperatorDomain`) so the gateway authenticates as the operator and the
+Historian records that operator; an empty password is allowed, and the password
+is scrubbed from any returned error. The streaming
 call returns a `StreamAlarmsClient`; cancel its context to terminate the
 stream. All three pass straight through to the gateway's central alarm
 monitor.
@@ -424,7 +428,7 @@ Every subcommand wired into the CLI. All accept the common flags
 | `bench-read-bulk` | Throughput benchmark (`-duration-seconds`, `-warmup-seconds`, `-bulk-size`). |
 | `stream-events` | Stream item-value events for a session (`-session-id`, `-limit`). |
 | `stream-alarms` | Stream the alarm feed (`-filter-prefix`, `-limit`). |
-| `acknowledge-alarm` | Acknowledge an alarm reference. |
+| `acknowledge-alarm` | Acknowledge an alarm as an operator (`-operator`, `-operator-domain`, `-operator-password-env`, default `MXGATEWAY_OPERATOR_PASSWORD`). |
 | `smoke` | End-to-end smoke workflow against one item. |
 | `galaxy-test-connection` | Probe the Galaxy Repository RPC connection. |
 | `galaxy-last-deploy` | Print the most recent deploy event. |

clients/go/cmd/mxgw-go/main.go

@@ -1119,6 +1119,10 @@ func formatAlarmFeedMessage(message *mxgateway.AlarmFeedMessage) string {
 	}
 }
 
+// defaultOperatorPasswordEnv is the CLI environment variable holding the
+// operator password for acknowledge-alarm.
+const defaultOperatorPasswordEnv = "MXGATEWAY_OPERATOR_PASSWORD"
+
 func runAcknowledgeAlarm(ctx context.Context, args []string, stdout, stderr io.Writer) error {
 	flags := flag.NewFlagSet("acknowledge-alarm", flag.ContinueOnError)
 	flags.SetOutput(stderr)
@@ -1126,7 +1130,12 @@ func runAcknowledgeAlarm(ctx context.Context, args []string, stdout, stderr io.W
 	jsonOutput := flags.Bool("json", false, "write JSON output")
 	reference := flags.String("reference", "", "full alarm reference to acknowledge")
 	comment := flags.String("comment", "", "operator acknowledge comment")
-	operator := flags.String("operator", "", "operator user performing the acknowledge")
+	operator := flags.String("operator", "", "operator user performing the acknowledge (required)")
+	operatorDomain := flags.String("operator-domain", "", "operator Galaxy domain (optional)")
+	// Prefer the environment variable so the credential stays out of shell
+	// history and the process table; an empty password is allowed.
+	operatorPassword := flags.String("operator-password", "", "operator password (prefer -operator-password-env)")
+	operatorPasswordEnv := flags.String("operator-password-env", defaultOperatorPasswordEnv, "environment variable containing the operator password")
 
 	if err := flags.Parse(args); err != nil {
 		return err
@@ -1134,6 +1143,17 @@ func runAcknowledgeAlarm(ctx context.Context, args []string, stdout, stderr io.W
 	if *reference == "" {
 		return errors.New("reference is required")
 	}
+	if *operator == "" {
+		return errors.New("operator is required")
+	}
+	resolvedOperatorPassword := *operatorPassword
+	if resolvedOperatorPassword == "" {
+		envName := *operatorPasswordEnv
+		if envName == "" {
+			envName = defaultOperatorPasswordEnv
+		}
+		resolvedOperatorPassword = os.Getenv(envName)
+	}
 
 	client, options, err := dialForCommand(ctx, common)
 	if err != nil {
@@ -1145,6 +1165,8 @@ func runAcknowledgeAlarm(ctx context.Context, args []string, stdout, stderr io.W
 		AlarmFullReference: *reference,
 		Comment:            *comment,
 		OperatorUser:       *operator,
+		OperatorDomain:     *operatorDomain,
+		OperatorPassword:   resolvedOperatorPassword,
 	})
 	if err != nil {
 		return err

clients/go/internal/generated/mxaccess_gateway.pb.go

@@ -3583,10 +3583,13 @@ func (x *AlarmFailoverConfig) GetFailbackStableProbes() int32 {
 	return 0
 }
 
-// Acknowledge a single alarm by its GUID. Operator identity fields are
+// Acknowledge a single alarm by its GUID. The worker resolves the GUID to the
-// recorded atomically with the ack transition in the alarm-history log.
+// alarm's Galaxy reference from its active-alarm snapshot and acknowledges it
-// The reply's hresult / native_status surfaces AVEVA's
+// by writing `comment` to `<TAGNAME>.AckMsg` over MXAccess after
-// AlarmAckByGUID return code.
+// AuthenticateUser as the operator, so the alarm history records the real
+// operator. The reply's hresult / native_status is 0 when the write was
+// applied (or sent but unconfirmed), otherwise the AuthenticateUser HRESULT or
+// the OnWriteComplete status detail.
 type AcknowledgeAlarmCommand struct {
 	state protoimpl.MessageState `protogen:"open.v1"`
 	// Canonical 8-4-4-4-12 GUID string (e.g. "BCC47053-9542-4D65-BDAA-BCDEA6A32A73").
@@ -3596,6 +3599,10 @@ type AcknowledgeAlarmCommand struct {
 	OperatorNode     string `protobuf:"bytes,4,opt,name=operator_node,json=operatorNode,proto3" json:"operator_node,omitempty"`
 	OperatorDomain   string `protobuf:"bytes,5,opt,name=operator_domain,json=operatorDomain,proto3" json:"operator_domain,omitempty"`
 	OperatorFullName string `protobuf:"bytes,6,opt,name=operator_full_name,json=operatorFullName,proto3" json:"operator_full_name,omitempty"`
+	// Operator credential used for AuthenticateUser before the AckMsg write.
+	// Implementations must keep this field out of logs, metrics labels, command
+	// lines, and diagnostics.
+	OperatorPassword string `protobuf:"bytes,7,opt,name=operator_password,json=operatorPassword,proto3" json:"operator_password,omitempty"`
 	unknownFields    protoimpl.UnknownFields
 	sizeCache        protoimpl.SizeCache
 }
@@ -3672,6 +3679,13 @@ func (x *AcknowledgeAlarmCommand) GetOperatorFullName() string {
 	return ""
 }
 
+func (x *AcknowledgeAlarmCommand) GetOperatorPassword() string {
+	if x != nil {
+		return x.OperatorPassword
+	}
+	return ""
+}
+
 // Snapshot the currently-active alarm set. Optional filter prefix scopes
 // the snapshot to alarms whose alarm_full_reference starts with the
 // supplied string (matches QueryActiveAlarmsRequest.alarm_filter_prefix).
@@ -3721,9 +3735,7 @@ func (x *QueryActiveAlarmsCommand) GetAlarmFilterPrefix() string {
 
 // Acknowledge a single alarm by its (name, provider, group) tuple. Used
 // when the public RPC's AlarmFullReference (Provider!Group.Tag) cannot
-// be resolved to a GUID directly. The worker invokes
+// be resolved to a GUID directly. The worker acknowledges by writing `comment` to `<alarm_name>.AckMsg` (Galaxy provider only) after AuthenticateUser as the operator.
-// wwAlarmConsumerClass.AlarmAckByName which reaches the same alarm
-// history path as AlarmAckByGUID.
 type AcknowledgeAlarmByNameCommand struct {
 	state protoimpl.MessageState `protogen:"open.v1"`
 	// Tag/alarm name (e.g. "TestMachine_001.TestAlarm001"). Tag itself
@@ -3739,6 +3751,10 @@ type AcknowledgeAlarmByNameCommand struct {
 	OperatorNode     string `protobuf:"bytes,6,opt,name=operator_node,json=operatorNode,proto3" json:"operator_node,omitempty"`
 	OperatorDomain   string `protobuf:"bytes,7,opt,name=operator_domain,json=operatorDomain,proto3" json:"operator_domain,omitempty"`
 	OperatorFullName string `protobuf:"bytes,8,opt,name=operator_full_name,json=operatorFullName,proto3" json:"operator_full_name,omitempty"`
+	// Operator credential used for AuthenticateUser before the AckMsg write.
+	// Implementations must keep this field out of logs, metrics labels, command
+	// lines, and diagnostics.
+	OperatorPassword string `protobuf:"bytes,9,opt,name=operator_password,json=operatorPassword,proto3" json:"operator_password,omitempty"`
 	unknownFields    protoimpl.UnknownFields
 	sizeCache        protoimpl.SizeCache
 }
@@ -3829,6 +3845,13 @@ func (x *AcknowledgeAlarmByNameCommand) GetOperatorFullName() string {
 	return ""
 }
 
+func (x *AcknowledgeAlarmByNameCommand) GetOperatorPassword() string {
+	if x != nil {
+		return x.OperatorPassword
+	}
+	return ""
+}
+
 type UnsubscribeBulkCommand struct {
 	state         protoimpl.MessageState `protogen:"open.v1"`
 	ServerHandle  int32                  `protobuf:"varint,1,opt,name=server_handle,json=serverHandle,proto3" json:"server_handle,omitempty"`
@@ -6127,11 +6150,11 @@ func (x *DrainEventsReply) GetEvents() []*MxEvent {
 // Reply payload for AcknowledgeAlarmCommand AND
 // AcknowledgeAlarmByNameCommand — both ack command kinds reuse this
 // payload case (`MxCommandReply.acknowledge_alarm`); there is no
-// dedicated by-name reply case. Surfaces AVEVA's native ack return
+// dedicated by-name reply case. Surfaces the ack outcome: 0 when the
-// code (AlarmAckByGUID for the GUID arm, AlarmAckByName for the
+// AckMsg write was applied or sent but unconfirmed; otherwise the
-// by-name arm); 0 means success. The MxCommandReply's hresult field
+// AuthenticateUser HRESULT or the OnWriteComplete status detail. The MxCommandReply's hresult field
 // carries the same value and is preferred for protocol consumers —
-// this payload exists so the gateway-side WorkerAlarmRpcDispatcher
+// this payload exists so GatewayAlarmMonitor
 // can echo native_status into AcknowledgeAlarmReply.hresult without
 // unpacking the outer envelope.
 type AcknowledgeAlarmReplyPayload struct {
@@ -7217,9 +7240,18 @@ type AcknowledgeAlarmRequest struct {
 	Comment string `protobuf:"bytes,4,opt,name=comment,proto3" json:"comment,omitempty"`
 	// Operator principal performing the acknowledgement. The lmxopcua side
 	// resolves this from the OPC UA session prior to invoking the RPC.
-	OperatorUser  string `protobuf:"bytes,5,opt,name=operator_user,json=operatorUser,proto3" json:"operator_user,omitempty"`
+	OperatorUser string `protobuf:"bytes,5,opt,name=operator_user,json=operatorUser,proto3" json:"operator_user,omitempty"`
-	unknownFields protoimpl.UnknownFields
+	// Operator's Galaxy (or OS) password. The gateway authenticates as
-	sizeCache     protoimpl.SizeCache
+	// operator_user with it before writing the acknowledgement so the alarm
+	// history records the real operator. May be empty for accounts without a
+	// password. Credential-sensitive: implementations must keep this field out
+	// of logs, metrics labels, command lines, and diagnostics.
+	OperatorPassword string `protobuf:"bytes,6,opt,name=operator_password,json=operatorPassword,proto3" json:"operator_password,omitempty"`
+	// Optional domain for OS-authenticated galaxies. When set the gateway
+	// authenticates as "<operator_domain>\<operator_user>".
+	OperatorDomain string `protobuf:"bytes,7,opt,name=operator_domain,json=operatorDomain,proto3" json:"operator_domain,omitempty"`
+	unknownFields  protoimpl.UnknownFields
+	sizeCache      protoimpl.SizeCache
 }
 
 func (x *AcknowledgeAlarmRequest) Reset() {
@@ -7280,14 +7312,27 @@ func (x *AcknowledgeAlarmRequest) GetOperatorUser() string {
 	return ""
 }
 
+func (x *AcknowledgeAlarmRequest) GetOperatorPassword() string {
+	if x != nil {
+		return x.OperatorPassword
+	}
+	return ""
+}
+
+func (x *AcknowledgeAlarmRequest) GetOperatorDomain() string {
+	if x != nil {
+		return x.OperatorDomain
+	}
+	return ""
+}
+
 type AcknowledgeAlarmReply struct {
 	state          protoimpl.MessageState `protogen:"open.v1"`
 	CorrelationId  string                 `protobuf:"bytes,2,opt,name=correlation_id,json=correlationId,proto3" json:"correlation_id,omitempty"`
 	ProtocolStatus *ProtocolStatus        `protobuf:"bytes,3,opt,name=protocol_status,json=protocolStatus,proto3" json:"protocol_status,omitempty"`
 	// Native ack return code echoed from the worker. The worker carries the
 	// ack outcome as a single int32 (AcknowledgeAlarmReplyPayload.native_status,
-	// = AlarmAckByName / AlarmAckByGUID return code; 0 = success); the gateway's
+	// 0 = applied or sent; otherwise the AuthenticateUser HRESULT or OnWriteComplete detail); GatewayAlarmMonitor copies that value here. This is the authoritative
-	// WorkerAlarmRpcDispatcher copies that value here. This is the authoritative
 	// ack-outcome field for the public RPC. Absent only when the worker reply
 	// omitted the value entirely (a protocol violation).
 	Hresult *int32 `protobuf:"varint,4,opt,name=hresult,proto3,oneof" json:"hresult,omitempty"`
@@ -8992,7 +9037,7 @@ const file_mxaccess_gateway_proto_rawDesc = "" +
 	"\x13AlarmFailoverConfig\x12B\n" +
 	"\x1dconsecutive_failure_threshold\x18\x01 \x01(\x05R\x1bconsecutiveFailureThreshold\x12E\n" +
 	"\x1ffailback_probe_interval_seconds\x18\x02 \x01(\x05R\x1cfailbackProbeIntervalSeconds\x124\n" +
-	"\x16failback_stable_probes\x18\x03 \x01(\x05R\x14failbackStableProbes\"\xf3\x01\n" +
+	"\x16failback_stable_probes\x18\x03 \x01(\x05R\x14failbackStableProbes\"\xa0\x02\n" +
 	"\x17AcknowledgeAlarmCommand\x12\x1d\n" +
 	"\n" +
 	"alarm_guid\x18\x01 \x01(\tR\talarmGuid\x12\x18\n" +
@@ -9000,9 +9045,10 @@ const file_mxaccess_gateway_proto_rawDesc = "" +
 	"\roperator_user\x18\x03 \x01(\tR\foperatorUser\x12#\n" +
 	"\roperator_node\x18\x04 \x01(\tR\foperatorNode\x12'\n" +
 	"\x0foperator_domain\x18\x05 \x01(\tR\x0eoperatorDomain\x12,\n" +
-	"\x12operator_full_name\x18\x06 \x01(\tR\x10operatorFullName\"J\n" +
+	"\x12operator_full_name\x18\x06 \x01(\tR\x10operatorFullName\x12+\n" +
+	"\x11operator_password\x18\a \x01(\tR\x10operatorPassword\"J\n" +
 	"\x18QueryActiveAlarmsCommand\x12.\n" +
-	"\x13alarm_filter_prefix\x18\x01 \x01(\tR\x11alarmFilterPrefix\"\xbd\x02\n" +
+	"\x13alarm_filter_prefix\x18\x01 \x01(\tR\x11alarmFilterPrefix\"\xea\x02\n" +
 	"\x1dAcknowledgeAlarmByNameCommand\x12\x1d\n" +
 	"\n" +
 	"alarm_name\x18\x01 \x01(\tR\talarmName\x12#\n" +
@@ -9013,7 +9059,8 @@ const file_mxaccess_gateway_proto_rawDesc = "" +
 	"\roperator_user\x18\x05 \x01(\tR\foperatorUser\x12#\n" +
 	"\roperator_node\x18\x06 \x01(\tR\foperatorNode\x12'\n" +
 	"\x0foperator_domain\x18\a \x01(\tR\x0eoperatorDomain\x12,\n" +
-	"\x12operator_full_name\x18\b \x01(\tR\x10operatorFullName\"`\n" +
+	"\x12operator_full_name\x18\b \x01(\tR\x10operatorFullName\x12+\n" +
+	"\x11operator_password\x18\t \x01(\tR\x10operatorPassword\"`\n" +
 	"\x16UnsubscribeBulkCommand\x12#\n" +
 	"\rserver_handle\x18\x01 \x01(\x05R\fserverHandle\x12!\n" +
 	"\fitem_handles\x18\x02 \x03(\x05R\vitemHandles\"v\n" +
@@ -9262,12 +9309,14 @@ const file_mxaccess_gateway_proto_rawDesc = "" +
 	"limitValue\x12\x1a\n" +
 	"\bdegraded\x18\x0e \x01(\bR\bdegraded\x12O\n" +
 	"\x0fsource_provider\x18\x0f \x01(\x0e2&.mxaccess_gateway.v1.AlarmProviderModeR\x0esourceProvider\x126\n" +
-	"\x17from_truncated_snapshot\x18\x10 \x01(\bR\x15fromTruncatedSnapshot\"\xd0\x01\n" +
+	"\x17from_truncated_snapshot\x18\x10 \x01(\bR\x15fromTruncatedSnapshot\"\xa6\x02\n" +
 	"\x17AcknowledgeAlarmRequest\x122\n" +
 	"\x15client_correlation_id\x18\x02 \x01(\tR\x13clientCorrelationId\x120\n" +
 	"\x14alarm_full_reference\x18\x03 \x01(\tR\x12alarmFullReference\x12\x18\n" +
 	"\acomment\x18\x04 \x01(\tR\acomment\x12#\n" +
-	"\roperator_user\x18\x05 \x01(\tR\foperatorUserJ\x04\b\x01\x10\x02R\n" +
+	"\roperator_user\x18\x05 \x01(\tR\foperatorUser\x12+\n" +
+	"\x11operator_password\x18\x06 \x01(\tR\x10operatorPassword\x12'\n" +
+	"\x0foperator_domain\x18\a \x01(\tR\x0eoperatorDomainJ\x04\b\x01\x10\x02R\n" +
 	"session_id\"\xb4\x02\n" +
 	"\x15AcknowledgeAlarmReply\x12%\n" +
 	"\x0ecorrelation_id\x18\x02 \x01(\tR\rcorrelationId\x12L\n" +

clients/go/mxgateway/alarms.go

@@ -10,6 +10,11 @@ import (
 // invoke:alarm-ack scope and forwards the acknowledge to the worker's MXAccess
 // session; the resulting native MxStatus is returned in the reply.
 //
+// OperatorUser is required: the gateway authenticates the operator (with
+// OperatorPassword and optional OperatorDomain) so the Historian records the
+// real operator. An empty password is allowed. The password is scrubbed from
+// any error this method returns.
+//
 // Acks are idempotent — re-acking an already-acked condition is a no-op at
 // the MxAccess layer.
 func (c *Client) AcknowledgeAlarm(ctx context.Context, req *AcknowledgeAlarmRequest) (*AcknowledgeAlarmReply, error) {
@@ -22,10 +27,10 @@ func (c *Client) AcknowledgeAlarm(ctx context.Context, req *AcknowledgeAlarmRequ
 
 	reply, err := c.raw.AcknowledgeAlarm(callCtx, req)
 	if err != nil {
-		return nil, &GatewayError{Op: "acknowledge alarm", Err: err}
+		return nil, redactSecrets(&GatewayError{Op: "acknowledge alarm", Err: err}, req.GetOperatorPassword())
 	}
 	if err := EnsureProtocolSuccess("acknowledge alarm", reply.GetProtocolStatus(), nil); err != nil {
-		return reply, err
+		return reply, redactSecrets(err, req.GetOperatorPassword())
 	}
 
 	return reply, nil

clients/go/mxgateway/alarms_test.go

@@ -5,6 +5,7 @@ import (
 	"errors"
 	"io"
 	"net"
+	"strings"
 	"testing"
 
 	pb "gitea.dohertylan.com/dohertj2/mxaccessgw/clients/go/internal/generated"
@@ -38,6 +39,8 @@ func TestAcknowledgeAlarmSendsRequestAndReturnsReply(t *testing.T) {
 		AlarmFullReference:  "Tank01.Level.HiHi",
 		Comment:             "investigating",
 		OperatorUser:        "alice",
+		OperatorPassword:    "pw-secret",
+		OperatorDomain:      "CORP",
 	})
 	if err != nil {
 		t.Fatalf("AcknowledgeAlarm() error = %v", err)
@@ -51,11 +54,37 @@ func TestAcknowledgeAlarmSendsRequestAndReturnsReply(t *testing.T) {
 	if got := fake.acknowledgeRequest.GetComment(); got != "investigating" {
 		t.Fatalf("captured comment = %q", got)
 	}
+	if got := fake.acknowledgeRequest.GetOperatorPassword(); got != "pw-secret" {
+		t.Fatalf("captured operator password mismatch")
+	}
+	if got := fake.acknowledgeRequest.GetOperatorDomain(); got != "CORP" {
+		t.Fatalf("captured operator domain = %q", got)
+	}
 	if got := fake.acknowledgeAuth; got != "Bearer test-api-key" {
 		t.Fatalf("authorization metadata = %q", got)
 	}
 }
 
+func TestAcknowledgeAlarmRedactsPasswordFromError(t *testing.T) {
+	fake := &fakeGatewayWithAlarms{
+		acknowledgeError: status.Error(codes.Internal, "boom pw-secret"),
+	}
+	client, cleanup := newBufconnClientWithAlarms(t, fake)
+	defer cleanup()
+
+	_, err := client.AcknowledgeAlarm(context.Background(), &pb.AcknowledgeAlarmRequest{
+		AlarmFullReference: "Tank01.Level.HiHi",
+		OperatorUser:       "alice",
+		OperatorPassword:   "pw-secret",
+	})
+	if err == nil {
+		t.Fatal("AcknowledgeAlarm() returned no error")
+	}
+	if strings.Contains(err.Error(), "pw-secret") {
+		t.Fatalf("error leaks operator password: %v", err)
+	}
+}
+
 func TestAcknowledgeAlarmRejectsNilRequest(t *testing.T) {
 	fake := &fakeGatewayWithAlarms{}
 	client, cleanup := newBufconnClientWithAlarms(t, fake)

clients/java/README.md

@@ -114,8 +114,11 @@ try (MxEventStream events = session.streamEventsAfter(lastSeenSequence)) {
 For alarms, `MxGatewayClient` exposes `queryActiveAlarms` (one-shot snapshot),
 `streamAlarms` (returns an `MxGatewayAlarmFeedSubscription` whose iterator
 yields alarm-feed messages from the gateway's central monitor), and
-`acknowledgeAlarm` (ack by full alarm reference with an optional comment and
+`acknowledgeAlarm` (ack by full alarm reference with an optional comment).
-ack target). Close the subscription to cancel the underlying gRPC stream.
+The ack carries operator credentials: `operator_user` is required, and
+`operator_password` / `operator_domain` let the gateway authenticate as that
+operator so the Historian records the real operator. An empty password is
+allowed. The password is scrubbed from exceptions the client throws. Close the subscription to cancel the underlying gRPC stream.
 
 `ActiveAlarmSnapshot.getFromTruncatedSnapshot()` reports that the record came
 from a provider fetch which hit the per-fetch cap: the snapshot set may omit
@@ -417,7 +420,7 @@ gradle :zb-mom-ww-mxgateway-cli:run --args="authenticate-user --endpoint localho
 gradle :zb-mom-ww-mxgateway-cli:run --args="write-secured --endpoint localhost:5000 --api-key-env MXGATEWAY_API_KEY --plaintext --session-id <id> --server-handle 1 --item-handle 1 --type int32 --value 123 --current-user-id 100 --verifier-user-id 100 --json"
 gradle :zb-mom-ww-mxgateway-cli:run --args="stream-events --endpoint localhost:5000 --api-key-env MXGATEWAY_API_KEY --plaintext --session-id <id> --limit 1 --json"
 gradle :zb-mom-ww-mxgateway-cli:run --args="stream-alarms --endpoint localhost:5000 --api-key-env MXGATEWAY_API_KEY --plaintext --filter-prefix Galaxy --limit 4 --json"
-gradle :zb-mom-ww-mxgateway-cli:run --args="acknowledge-alarm --endpoint localhost:5000 --api-key-env MXGATEWAY_API_KEY --plaintext --reference \"\\Galaxy\Area001.Pump001.PumpFault\" --json"
+gradle :zb-mom-ww-mxgateway-cli:run --args="acknowledge-alarm --endpoint localhost:5000 --api-key-env MXGATEWAY_API_KEY --plaintext --reference \"\\Galaxy\Area001.Pump001.PumpFault\" --operator operator1 --operator-password-env MXGATEWAY_OPERATOR_PASSWORD --json"
 gradle :zb-mom-ww-mxgateway-cli:run --args="smoke --endpoint localhost:5000 --api-key-env MXGATEWAY_API_KEY --plaintext --item TestObject.TestInt --json"
 ```
 

clients/java/zb-mom-ww-mxgateway-cli/src/main/java/com/zb/mom/ww/mxgateway/cli/MxGatewayCli.java

@@ -1822,6 +1822,31 @@ public final class MxGatewayCli implements Callable<Integer> {
         @Option(names = "--operator", description = "Operator user performing the acknowledge.")
         String operator = "";
 
+        @Option(names = "--operator-domain", description = "Operator Galaxy domain (optional).")
+        String operatorDomain = "";
+
+        @Option(
+                names = "--operator-password",
+                description = "Operator Galaxy password (credential; prefer --operator-password-env). Never echoed.")
+        String operatorPassword;
+
+        @Option(
+                names = "--operator-password-env",
+                defaultValue = "MXGATEWAY_OPERATOR_PASSWORD",
+                description = "Environment variable holding the operator password when --operator-password is omitted.")
+        String operatorPasswordEnv;
+
+        private String resolveOperatorPassword() {
+            if (operatorPassword != null && !operatorPassword.isEmpty()) {
+                return operatorPassword;
+            }
+            String name = operatorPasswordEnv == null || operatorPasswordEnv.isBlank()
+                    ? "MXGATEWAY_OPERATOR_PASSWORD"
+                    : operatorPasswordEnv;
+            String value = System.getenv(name);
+            return value == null ? "" : value;
+        }
+
         AcknowledgeAlarmCommand(MxGatewayCliClientFactory clientFactory) {
             super(clientFactory);
         }
@@ -1833,6 +1858,8 @@ public final class MxGatewayCli implements Callable<Integer> {
                         .setAlarmFullReference(reference)
                         .setComment(comment)
                         .setOperatorUser(operator)
+                        .setOperatorDomain(operatorDomain == null ? "" : operatorDomain)
+                        .setOperatorPassword(resolveOperatorPassword())
                         .build());
                 writeOutput(
                         "acknowledge-alarm",

clients/java/zb-mom-ww-mxgateway-cli/src/test/java/com/zb/mom/ww/mxgateway/cli/MxGatewayCliTests.java

@@ -714,9 +714,16 @@ final class MxGatewayCliTests {
                 "checked",
                 "--operator",
                 "operator1",
+                "--operator-domain",
+                "CORP",
+                "--operator-password",
+                "pw-secret",
                 "--json");
 
         assertEquals(0, run.exitCode());
+        assertEquals("CORP", factory.client.lastAcknowledgeAlarmRequest.getOperatorDomain());
+        assertEquals("pw-secret", factory.client.lastAcknowledgeAlarmRequest.getOperatorPassword());
+        assertFalse(run.output().contains("pw-secret"), run.output());
         assertEquals("Tank01.Level.HiHi", factory.client.lastAcknowledgeAlarmRequest.getAlarmFullReference());
         assertEquals("checked", factory.client.lastAcknowledgeAlarmRequest.getComment());
         assertEquals("operator1", factory.client.lastAcknowledgeAlarmRequest.getOperatorUser());
@@ -770,6 +777,10 @@ final class MxGatewayCliTests {
                 "--plaintext",
                 "--reference",
                 "\\Galaxy\\Area001.Pump001.PumpFault",
+                "--operator",
+                "operator1",
+                "--operator-password-env",
+                "MXGATEWAY_OPERATOR_PASSWORD",
                 "--json"
         };
         assertReadmeExampleParses(args);

clients/java/zb-mom-ww-mxgateway-client/src/main/java/com/zb/mom/ww/mxgateway/client/MxGatewayClient.java

@@ -273,6 +273,11 @@ public final class MxGatewayClient implements AutoCloseable {
      * worker's MXAccess session; the resulting native MxStatus is returned
      * in the reply. Acks are idempotent at the MxAccess layer.
      *
+     * <p>{@code operator_user} is required; the gateway authenticates it with
+     * {@code operator_password} (and optional {@code operator_domain}) and the
+     * Historian records that operator. An empty password is allowed. The
+     * password is scrubbed from any exception this method throws.
+     *
      * @param request the {@code AcknowledgeAlarmRequest}
      * @return the raw acknowledge reply
      * @throws MxGatewayException on transport or protocol failure
@@ -283,10 +288,14 @@ public final class MxGatewayClient implements AutoCloseable {
             MxGatewayErrors.ensureProtocolSuccess("acknowledge alarm", reply.getProtocolStatus(), null);
             return reply;
         } catch (RuntimeException error) {
-            if (error instanceof MxGatewayException) {
+            MxGatewayException mapped = error instanceof MxGatewayException gatewayError
-                throw error;
+                    ? gatewayError
+                    : (MxGatewayException) MxGatewayErrors.fromGrpc("acknowledge alarm", error);
+            String secret = request.getOperatorPassword();
+            if (secret.isEmpty()) {
+                throw mapped;
             }
-            throw MxGatewayErrors.fromGrpc("acknowledge alarm", error);
+            throw MxGatewayErrors.redacted(mapped, secret);
         }
     }
 
@@ -299,10 +308,24 @@ public final class MxGatewayClient implements AutoCloseable {
      */
     public CompletableFuture<AcknowledgeAlarmReply> acknowledgeAlarmAsync(AcknowledgeAlarmRequest request) {
         CompletableFuture<AcknowledgeAlarmReply> future = toCompletable(rawFutureStub().acknowledgeAlarm(request));
-        return future.thenApply(reply -> {
+        String secret = request.getOperatorPassword();
+        CompletableFuture<AcknowledgeAlarmReply> checked = future.thenApply(reply -> {
             MxGatewayErrors.ensureProtocolSuccess("acknowledge alarm", reply.getProtocolStatus(), null);
             return reply;
         });
+        if (secret.isEmpty()) {
+            return checked;
+        }
+        // Scrub the operator password from any failure surfaced through the future.
+        return checked.exceptionally(error -> {
+            Throwable cause = error instanceof java.util.concurrent.CompletionException && error.getCause() != null
+                    ? error.getCause()
+                    : error;
+            if (cause instanceof MxGatewayException gatewayError) {
+                throw new java.util.concurrent.CompletionException(MxGatewayErrors.redacted(gatewayError, secret));
+            }
+            throw new java.util.concurrent.CompletionException(cause);
+        });
     }
 
     /**

clients/java/zb-mom-ww-mxgateway-client/src/main/java/com/zb/mom/ww/mxgateway/client/MxGatewayErrors.java

@@ -28,6 +28,43 @@ final class MxGatewayErrors {
         return new MxGatewayException("gateway " + operation + " failed: " + error.getMessage(), error);
     }
 
+    /**
+     * Rebuilds a gateway exception with an exact secret scrubbed from its
+     * message and protocol status. The original is not chained as a cause
+     * because its message and cause chain may carry the secret.
+     */
+    static MxGatewayException redacted(MxGatewayException error, String secret) {
+        String message = MxGatewaySecrets.redactExact(error.getMessage(), secret);
+        if (error instanceof MxAccessException ex) {
+            return new MxAccessException(message, scrub(ex.protocolStatus(), secret), ex.reply(), null);
+        }
+        if (error instanceof MxGatewayCommandException ex) {
+            return new MxGatewayCommandException(message, scrub(ex.protocolStatus(), secret), ex.reply(), null);
+        }
+        if (error instanceof MxGatewaySessionException ex) {
+            return new MxGatewaySessionException(message, scrub(ex.protocolStatus(), secret), null);
+        }
+        if (error instanceof MxGatewayWorkerException ex) {
+            return new MxGatewayWorkerException(message, scrub(ex.protocolStatus(), secret), null);
+        }
+        if (error instanceof MxGatewayAuthenticationException) {
+            return new MxGatewayAuthenticationException(message, null);
+        }
+        if (error instanceof MxGatewayAuthorizationException) {
+            return new MxGatewayAuthorizationException(message, null);
+        }
+        return new MxGatewayException(message);
+    }
+
+    private static ProtocolStatus scrub(ProtocolStatus status, String secret) {
+        if (status == null) {
+            return null;
+        }
+        return status.toBuilder()
+                .setMessage(MxGatewaySecrets.redactExact(status.getMessage(), secret))
+                .build();
+    }
+
     static void ensureProtocolSuccess(String operation, ProtocolStatus status, MxCommandReply reply) {
         if (status == null || status.getCode() == ProtocolStatusCode.PROTOCOL_STATUS_CODE_OK) {
             return;

clients/java/zb-mom-ww-mxgateway-client/src/test/java/com/zb/mom/ww/mxgateway/client/MxGatewayCredentialReplyTests.java

@@ -74,6 +74,52 @@ final class MxGatewayCredentialReplyTests {
         }
     }
 
+    @Test
+    void acknowledgeAlarmAsyncRedactsOperatorPasswordFromFailure() throws Exception {
+        String serverName = "mxgw-java-ack-" + UUID.randomUUID();
+        MxAccessGatewayGrpc.MxAccessGatewayImplBase service = new MxAccessGatewayGrpc.MxAccessGatewayImplBase() {
+            @Override
+            public void acknowledgeAlarm(
+                    mxaccess_gateway.v1.MxaccessGateway.AcknowledgeAlarmRequest request,
+                    StreamObserver<mxaccess_gateway.v1.MxaccessGateway.AcknowledgeAlarmReply> observer) {
+                observer.onNext(mxaccess_gateway.v1.MxaccessGateway.AcknowledgeAlarmReply.newBuilder()
+                        .setProtocolStatus(ProtocolStatus.newBuilder()
+                                .setCode(ProtocolStatusCode.PROTOCOL_STATUS_CODE_MXACCESS_FAILURE)
+                                .setMessage("auth failed for operator1 password " + CREDENTIAL))
+                        .build());
+                observer.onCompleted();
+            }
+        };
+        Server server = InProcessServerBuilder.forName(serverName)
+                .directExecutor().addService(service).build().start();
+        ManagedChannel channel = InProcessChannelBuilder.forName(serverName).directExecutor().build();
+        try (MxGatewayClient client = new MxGatewayClient(
+                channel,
+                MxGatewayClientOptions.builder()
+                        .endpoint("in-process").apiKey("").plaintext(true)
+                        .callTimeout(Duration.ofSeconds(5)).build())) {
+            var request = mxaccess_gateway.v1.MxaccessGateway.AcknowledgeAlarmRequest.newBuilder()
+                    .setAlarmFullReference("Tank01.Level.HiHi")
+                    .setOperatorUser("operator1")
+                    .setOperatorPassword(CREDENTIAL)
+                    .build();
+
+            java.util.concurrent.ExecutionException async = assertThrows(
+                    java.util.concurrent.ExecutionException.class,
+                    () -> client.acknowledgeAlarmAsync(request).get());
+            assertTrue(async.getCause() instanceof MxGatewayException, String.valueOf(async.getCause()));
+            assertFalse(async.getCause().getMessage().contains(CREDENTIAL));
+            assertFalse(((MxGatewayCommandException) async.getCause()).protocolStatus().getMessage()
+                    .contains(CREDENTIAL));
+
+            MxGatewayException sync = assertThrows(MxGatewayException.class, () -> client.acknowledgeAlarm(request));
+            assertFalse(sync.getMessage().contains(CREDENTIAL));
+        } finally {
+            channel.shutdownNow();
+            server.shutdownNow();
+        }
+    }
+
     @Test
     void authenticateUserMissingPayloadThrowsMalformedReply() throws Exception {
         MxCommandReply reply = loadReply("authenticate-user.missing-payload.reply.json");

clients/python/README.md

@@ -111,9 +111,15 @@ MXAccess commands and preserve raw replies on typed command exceptions.
 For alarms, the client exposes `GatewayClient.query_active_alarms` (one-shot
 snapshot), `GatewayClient.stream_alarms` (async generator yielding alarm-feed
 messages from the gateway's central monitor), and
-`GatewayClient.acknowledge_alarm` (ack by alarm reference, optional comment
+`GatewayClient.acknowledge_alarm` (ack by alarm reference with an optional
-and ack target). Cancel the surrounding task or `aclose()` the iterator to
+comment). The ack is made as the operator: set `operator_user` (required) and
-terminate the stream.
+`operator_password` (may be empty) and optionally `operator_domain` on
+`AcknowledgeAlarmRequest`. The gateway authenticates the operator and writes the
+comment to the alarm's `AckMsg`, so the alarm history records the operator. The
+password is redacted from any raised error.
+
+Cancel the surrounding task or `aclose()` an alarm iterator to terminate the
+stream.
 
 `ActiveAlarmSnapshot.from_truncated_snapshot` reports that the record came from
 a provider fetch which hit the per-fetch cap: the snapshot set may omit active
@@ -457,7 +463,7 @@ mxgw-py add-item --session-id <id> --server-handle 1 --item Object.Attribute --j
 mxgw-py advise --session-id <id> --server-handle 1 --item-handle 2 --json
 mxgw-py stream-events --session-id <id> --max-events 1 --json
 mxgw-py stream-alarms --max-messages 4 --json
-mxgw-py acknowledge-alarm --reference "\\Galaxy\Area001.Pump001.PumpFault" --json
+mxgw-py acknowledge-alarm --reference "\\Galaxy\Area001.Pump001.PumpFault" --operator operator1 --operator-password-env MXGATEWAY_OPERATOR_PASSWORD --json
 mxgw-py write --session-id <id> --server-handle 1 --item-handle 2 --type int32 --value 123 --json
 ```
 

clients/python/src/zb_mom_ww_mxgateway/client.py

@@ -9,7 +9,7 @@ from typing import Any
 import grpc
 
 from .auth import merge_metadata
-from .errors import ensure_protocol_success, map_rpc_error
+from .errors import MxGatewayError, ensure_protocol_success, map_rpc_error
 from .generated import mxaccess_gateway_pb2 as pb
 from .generated import mxaccess_gateway_pb2_grpc as pb_grpc
 from .options import ClientOptions, create_channel, resolve_channel_security
@@ -148,17 +148,23 @@ class GatewayClient:
         self,
         request: pb.AcknowledgeAlarmRequest,
     ) -> pb.AcknowledgeAlarmReply:
-        """Acknowledge an active MXAccess alarm condition through the gateway.
+        """Acknowledge an alarm as the operator.
 
-        The gateway authenticates the request against the API key's
+        The gateway authenticates ``operator_user`` with ``operator_password`` and
-        ``invoke:alarm-ack`` scope and forwards the acknowledge to the worker's
+        writes the comment to the alarm's ``AckMsg``, so the alarm history records
-        MXAccess session; the resulting native ``MxStatus`` is returned in the
+        the operator. The password is redacted from any raised error.
-        reply. Acks are idempotent — re-acking an already-acked condition is a
-        no-op at the MxAccess layer.
         """
-        reply = await self._unary("acknowledge alarm", self.raw_stub.AcknowledgeAlarm, request)
+        try:
-        ensure_protocol_success("acknowledge alarm", reply.protocol_status, reply)
+            reply = await self._unary("acknowledge alarm", self.raw_stub.AcknowledgeAlarm, request)
-        return reply
+            ensure_protocol_success("acknowledge alarm", reply.protocol_status, reply)
+            return reply
+        except MxGatewayError as error:
+            if request.operator_password:
+                # Imported lazily: session.py imports this module at its tail.
+                from .session import _redact_error
+
+                _redact_error(error, [request.operator_password])
+            raise
 
     def query_active_alarms(
         self,

clients/python/src/zb_mom_ww_mxgateway/session.py

@@ -3,6 +3,7 @@
 from __future__ import annotations
 
 from collections.abc import AsyncIterator, Sequence
+from typing import Any
 
 from .auth import redact_secret
 from .errors import MalformedReplyError, MxGatewayError, ensure_mxaccess_success
@@ -943,22 +944,26 @@ def _redact_error(error: MxGatewayError, secrets: Sequence[str | None]) -> None:
         error.raw_reply = _redact_reply(error.raw_reply, scrubbed)
 
 
-def _redact_reply(reply: pb.MxCommandReply, secrets: Sequence[str]) -> pb.MxCommandReply:
+def _redact_reply(reply: Any, secrets: Sequence[str]) -> Any:
     """Return a deep copy of *reply* with credential text scrubbed from diagnostics.
 
     Operates on a clone so the caller's original reply object is never mutated.
     Only the free-text diagnostic fields that can echo a client-supplied secret
     are scrubbed; the structured/enum fields the gateway itself sets are left as-is.
+    Works for any reply message (``MxCommandReply``, ``AcknowledgeAlarmReply``, ...):
+    each field is scrubbed only when the message type declares it.
     """
     clone = type(reply)()
     clone.CopyFrom(reply)
-    if clone.protocol_status.message:
+    fields = clone.DESCRIPTOR.fields_by_name
+    if "protocol_status" in fields and clone.protocol_status.message:
         clone.protocol_status.message = redact_secret(clone.protocol_status.message, secrets)
-    if clone.diagnostic_message:
+    if "diagnostic_message" in fields and clone.diagnostic_message:
         clone.diagnostic_message = redact_secret(clone.diagnostic_message, secrets)
-    for status in clone.statuses:
+    if "statuses" in fields:
-        if status.diagnostic_text:
+        for status in clone.statuses:
-            status.diagnostic_text = redact_secret(status.diagnostic_text, secrets)
+            if status.diagnostic_text:
+                status.diagnostic_text = redact_secret(status.diagnostic_text, secrets)
     return clone
 
 

clients/python/src/zb_mom_ww_mxgateway_cli/commands.py

@@ -36,6 +36,7 @@ MAX_AGGREGATE_EVENTS = 10_000
 #: CLI (CLI-45) so one exported variable drives the same operator workflow in all
 #: five languages.
 DEFAULT_VERIFY_PASSWORD_ENV = "MXGATEWAY_VERIFY_PASSWORD"
+DEFAULT_OPERATOR_PASSWORD_ENV = "MXGATEWAY_OPERATOR_PASSWORD"
 
 _BATCH_EOR = "__MXGW_BATCH_EOR__"
 
@@ -528,16 +529,30 @@ def stream_alarms(**kwargs: Any) -> None:
 @gateway_options
 @click.option("--reference", required=True, help="Alarm full reference to acknowledge.")
 @click.option("--comment", default="", help="Acknowledgement comment.")
-@click.option("--operator", default="", help="Operator user name.")
+@click.option("--operator", required=True, help="Operator user name (recorded in the alarm history).")
+@click.option("--operator-domain", default="", help="Operator Windows/Galaxy domain (optional).")
+@click.option(
+    "--operator-password",
+    default=None,
+    help="Operator password. Prefer --operator-password-env so the secret is not visible on the command line.",
+)
+@click.option(
+    "--operator-password-env",
+    default=DEFAULT_OPERATOR_PASSWORD_ENV,
+    show_default=True,
+    help="Environment variable holding the operator password (an unset variable means an empty password).",
+)
 @click.option("--correlation-id", default="", help="Client correlation id.")
 @click.option("--json", "output_json", is_flag=True, help="Emit JSON output.")
 def acknowledge_alarm(**kwargs: Any) -> None:
-    """Acknowledge an active MXAccess alarm condition (session-less)."""
+    """Acknowledge an active MXAccess alarm condition as the operator (session-less)."""
 
+    password = _resolve_operator_password(kwargs)
+    kwargs["operator_password"] = password
     _run(
         _acknowledge_alarm(**kwargs),
         output_json=kwargs["output_json"],
-        secrets=_secrets(kwargs),
+        secrets=_secrets(kwargs) + [password],
     )
 
 
@@ -836,6 +851,21 @@ async def _authenticate_user(**kwargs: Any) -> dict[str, Any]:
         return {"userId": user_id}
 
 
+def _resolve_operator_password(kwargs: dict[str, Any]) -> str:
+    """Resolve the alarm-ack operator password from --operator-password or its env var.
+
+    Unlike :func:`_resolve_password`, an unset or empty value is allowed: Galaxy
+    accounts may have empty passwords. The value is never echoed; callers add it
+    to the ``secrets`` redaction list.
+    """
+
+    env_name = kwargs.get("operator_password_env") or DEFAULT_OPERATOR_PASSWORD_ENV
+    password = kwargs.get("operator_password")
+    if password is None:
+        password = os.environ.get(env_name, "")
+    return password
+
+
 def _resolve_password(kwargs: dict[str, Any]) -> str:
     """Resolve the authenticate-user password from --password or --password-env.
 
@@ -1161,6 +1191,8 @@ async def _acknowledge_alarm(**kwargs: Any) -> dict[str, Any]:
                 alarm_full_reference=kwargs["reference"],
                 comment=kwargs["comment"],
                 operator_user=kwargs["operator"],
+                operator_domain=kwargs["operator_domain"],
+                operator_password=kwargs["operator_password"],
             ),
         )
         return _message_dict(reply)

clients/python/tests/test_alarms.py

@@ -34,6 +34,8 @@ async def test_acknowledge_alarm_sends_request_and_returns_reply() -> None:
             alarm_full_reference="Tank01.Level.HiHi",
             comment="investigating",
             operator_user="alice",
+            operator_password="pw-secret",
+            operator_domain="CORP",
         ),
     )
 
@@ -44,9 +46,68 @@ async def test_acknowledge_alarm_sends_request_and_returns_reply() -> None:
     assert captured_request.alarm_full_reference == "Tank01.Level.HiHi"
     assert captured_request.comment == "investigating"
     assert captured_request.operator_user == "alice"
+    assert captured_request.operator_password == "pw-secret"
+    assert captured_request.operator_domain == "CORP"
     assert stub.acknowledge_alarm.metadata == (("authorization", "Bearer mxgw_test_secret"),)
 
 
+@pytest.mark.asyncio
+async def test_acknowledge_alarm_error_redacts_operator_password() -> None:
+    stub = FakeGatewayStub()
+    stub.acknowledge_alarm.exception = FakeRpcError(
+        grpc.StatusCode.UNAUTHENTICATED, "bad credentials pw-secret",
+    )
+    client = await GatewayClient.connect(
+        ClientOptions(endpoint="fake", api_key="mxgw_test_secret", plaintext=True),
+        stub=stub,
+    )
+
+    with pytest.raises(MxGatewayAuthenticationError) as excinfo:
+        await client.acknowledge_alarm(
+            pb.AcknowledgeAlarmRequest(
+                alarm_full_reference="Tank01.Level.HiHi",
+                operator_user="alice",
+                operator_password="pw-secret",
+            ),
+        )
+
+    assert "pw-secret" not in str(excinfo.value)
+
+
+@pytest.mark.asyncio
+async def test_acknowledge_alarm_protocol_failure_redacts_operator_password() -> None:
+    from zb_mom_ww_mxgateway.errors import MxGatewayError
+
+    stub = FakeGatewayStub()
+    stub.acknowledge_alarm.replies = [
+        pb.AcknowledgeAlarmReply(
+            protocol_status=pb.ProtocolStatus(
+                code=pb.PROTOCOL_STATUS_CODE_INVALID_REQUEST,
+                message="rejected credential pw-secret",
+            ),
+        ),
+    ]
+    client = await GatewayClient.connect(
+        ClientOptions(endpoint="fake", api_key="mxgw_test_secret", plaintext=True),
+        stub=stub,
+    )
+
+    with pytest.raises(MxGatewayError) as excinfo:
+        await client.acknowledge_alarm(
+            pb.AcknowledgeAlarmRequest(
+                alarm_full_reference="Tank01.Level.HiHi",
+                operator_user="alice",
+                operator_password="pw-secret",
+            ),
+        )
+
+    assert "pw-secret" not in str(excinfo.value)
+    raw = excinfo.value.raw_reply
+    assert isinstance(raw, pb.AcknowledgeAlarmReply)
+    assert "pw-secret" not in raw.protocol_status.message
+    assert "pw-secret" not in str(raw)
+
+
 @pytest.mark.asyncio
 async def test_acknowledge_alarm_unauthenticated_raises_typed_error() -> None:
     stub = FakeGatewayStub()

clients/python/tests/test_review_findings_022_to_026.py

@@ -771,6 +771,7 @@ def test_cli_stream_alarms_positive_max_messages_remains_finite(monkeypatch) ->
 
 
 def test_cli_acknowledge_alarm_happy_path(monkeypatch) -> None:
+    monkeypatch.setenv("MXGW_TEST_OPPW", "pw-secret")
     stub = _BulkFakeStub()
     stub.acknowledge_alarm = _BulkFakeUnary(
         [
@@ -798,12 +799,19 @@ def test_cli_acknowledge_alarm_happy_path(monkeypatch) -> None:
             "investigating",
             "--operator",
             "alice",
+            "--operator-domain",
+            "CORP",
+            "--operator-password-env",
+            "MXGW_TEST_OPPW",
             "--json",
         ],
     )
 
     assert result.exit_code == 0, result.output
+    assert "pw-secret" not in result.output
     captured_request = stub.acknowledge_alarm.requests[0]
+    assert captured_request.operator_domain == "CORP"
+    assert captured_request.operator_password == "pw-secret"
     assert captured_request.alarm_full_reference == "Tank01.Level.HiHi"
     assert captured_request.comment == "investigating"
     assert captured_request.operator_user == "alice"

clients/rust/README.md

@@ -78,7 +78,8 @@ cargo run -p mxgw-cli -- add-item --session-id <session-id> --server-handle 1 --
 cargo run -p mxgw-cli -- advise --session-id <session-id> --server-handle 1 --item-handle 1 --json
 cargo run -p mxgw-cli -- stream-events --session-id <session-id> --max-events 1 --json
 cargo run -p mxgw-cli -- stream-alarms --session-id <session-id> --max-messages 4 --json
-cargo run -p mxgw-cli -- acknowledge-alarm --session-id <session-id> --alarm-reference "\\Galaxy\Area001.Pump001.PumpFault" --json
+cargo run -p mxgw-cli -- acknowledge-alarm --reference "Area001.Pump001.PumpFault" --operator <user> [--operator-domain <domain>] --json
+# operator password: MXGATEWAY_OPERATOR_PASSWORD, or --operator-password-env / --operator-password
 cargo run -p mxgw-cli -- write --session-id <session-id> --server-handle 1 --item-handle 1 --value-type int32 --value 123 --json
 ```
 
@@ -124,6 +125,9 @@ creates an authenticated `tonic` client and attaches `authorization: Bearer
 `stream_alarms`, `acknowledge_alarm`, and `raw_client`. `stream_alarms`
 returns an `AlarmFeedStream` async stream of alarm-feed messages and
 shares the gateway's central alarm monitor with every other client.
+`acknowledge_alarm` requires `operator_user` and carries `operator_password` (may be
+empty) and `operator_domain`; the gateway authenticates as that operator so the Historian
+records them. The password is scrubbed from any returned error's `Display`/`Debug`.
 
 The `from_truncated_snapshot` field on `ActiveAlarmSnapshot` reports that the record came from
 a provider fetch which hit the per-fetch cap: the snapshot set may omit active

clients/rust/RustClientDesign.md

@@ -174,6 +174,8 @@ alarm monitor — no worker session is opened — so any number of clients may
 attach. Dropping the stream cancels the gRPC call cooperatively.
 `acknowledge_alarm` is idempotent at the MxAccess layer; the returned
 `AcknowledgeAlarmReply` carries the native MxStatus from the worker.
+`operator_user` is required; `operator_password` may be empty and is scrubbed from any
+returned error via `Error::redact_secrets`.
 
 ## Galaxy Repository
 
@@ -346,7 +348,7 @@ mxgw write-secured-bulk ...
 mxgw write-secured2-bulk ...
 mxgw stream-events --session-id <id> --json
 mxgw stream-alarms [--filter-prefix <prefix>] [--max-events <n>]
-mxgw acknowledge-alarm --reference <full-ref> [--comment <txt>] [--operator <user>]
+mxgw acknowledge-alarm --reference <full-ref> [--comment <txt>] --operator <user> [--operator-domain <domain>] [--operator-password-env <VAR>=MXGATEWAY_OPERATOR_PASSWORD] [--operator-password <pw>]
 mxgw bench-read-bulk [--duration-seconds <n>] [--warmup-seconds <n>] [--bulk-size <n>]
 mxgw smoke --endpoint http://localhost:5000 --api-key-env MXGATEWAY_API_KEY --item TestChildObject.TestInt
 mxgw batch

clients/rust/crates/mxgw-cli/src/main.rs

@@ -374,8 +374,19 @@ enum Command {
         reference: String,
         #[arg(long, default_value = "")]
         comment: String,
-        #[arg(long, default_value = "")]
+        /// Galaxy account of the acknowledging operator (required; the gateway
+        /// rejects an empty operator so the real operator is recorded).
+        #[arg(long)]
         operator: String,
+        /// Galaxy domain of the operator; empty authenticates the bare user name.
+        #[arg(long, default_value = "")]
+        operator_domain: String,
+        /// Operator password (prefer `--operator-password-env`; empty is allowed).
+        #[arg(long)]
+        operator_password: Option<String>,
+        /// Environment variable holding the operator password.
+        #[arg(long, default_value = "MXGATEWAY_OPERATOR_PASSWORD")]
+        operator_password_env: String,
         #[arg(long)]
         json: bool,
     },
@@ -1081,8 +1092,16 @@ async fn dispatch(command: Command) -> Result<(), Error> {
             reference,
             comment,
             operator,
+            operator_domain,
+            operator_password,
+            operator_password_env,
             json,
         } => {
+            // An empty password is allowed (Galaxy accounts may have one), so
+            // an unset flag and env resolve to an empty string.
+            let operator_password = operator_password
+                .or_else(|| env::var(&operator_password_env).ok())
+                .unwrap_or_default();
             let client = connect(connection).await?;
             let reply = client
                 .acknowledge_alarm(AcknowledgeAlarmRequest {
@@ -1090,6 +1109,8 @@ async fn dispatch(command: Command) -> Result<(), Error> {
                     alarm_full_reference: reference,
                     comment,
                     operator_user: operator,
+                    operator_password,
+                    operator_domain,
                 })
                 .await?;
             print_acknowledge_alarm_reply(&reply, json);
@@ -2771,6 +2792,17 @@ mod tests {
         assert!(parsed.is_ok(), "parse failed: {parsed:?}");
     }
 
+    #[test]
+    fn acknowledge_alarm_requires_operator() {
+        let parsed = Cli::try_parse_from([
+            "mxgw",
+            "acknowledge-alarm",
+            "--reference",
+            "Tank01.Level.HiHi",
+        ]);
+        assert!(parsed.is_err());
+    }
+
     #[test]
     fn acknowledge_alarm_requires_reference() {
         let parsed = Cli::try_parse_from(["mxgw", "acknowledge-alarm"]);

clients/rust/src/client.rs

@@ -327,6 +327,9 @@ impl GatewayClient {
     /// MXAccess session; the resulting native MxStatus is returned in the
     /// reply. Acks are idempotent at the MxAccess layer.
     ///
+    /// `operator_user` is required; `operator_password` may be empty. The
+    /// password is scrubbed from any returned error's `Display`/`Debug`.
+    ///
     /// # Errors
     ///
     /// Returns [`Error::ProtocolStatus`] when the gateway accepts the call but
@@ -336,13 +339,19 @@ impl GatewayClient {
         &self,
         request: AcknowledgeAlarmRequest,
     ) -> Result<AcknowledgeAlarmReply, Error> {
+        let secret = request.operator_password.clone();
         let mut client = self.inner.clone();
-        let response = client
+        let result = async {
-            .acknowledge_alarm(self.unary_request(request))
+            let response = client
-            .await?;
+                .acknowledge_alarm(self.unary_request(request))
-        let reply = response.into_inner();
+                .await?;
-        ensure_protocol_success("acknowledge alarm", reply.protocol_status.as_ref())?;
+            let reply = response.into_inner();
-        Ok(reply)
+            ensure_protocol_success("acknowledge alarm", reply.protocol_status.as_ref())?;
+            Ok(reply)
+        }
+        .await;
+        // The operator password must never surface in a returned error.
+        result.map_err(|error: Error| error.redact_secrets(&[secret.as_str()]))
     }
 
     /// Open the server-streaming `QueryActiveAlarms` RPC — the gateway's

clients/rust/src/error.rs

@@ -314,6 +314,68 @@ impl std::fmt::Display for MxAccessError {
 
 impl std::error::Error for MxAccessError {}
 
+impl Error {
+    /// Scrub exact caller-supplied secrets (for example an operator password)
+    /// from the error's rendered text and any wrapped `tonic::Status`, so that
+    /// neither `Display` nor `Debug` can reveal them. Empty secrets are ignored.
+    #[must_use]
+    pub fn redact_secrets(self, secrets: &[&str]) -> Self {
+        let scrub = |text: &str| {
+            let mut out = text.to_owned();
+            for secret in secrets.iter().filter(|secret| !secret.is_empty()) {
+                out = out.replace(*secret, "<redacted>");
+            }
+            out
+        };
+        let scrub_status = |status: Box<tonic::Status>| {
+            Box::new(tonic::Status::new(status.code(), scrub(status.message())))
+        };
+        match self {
+            Self::Authentication { message, status } => Self::Authentication {
+                message: scrub(&message),
+                status: scrub_status(status),
+            },
+            Self::Authorization { message, status } => Self::Authorization {
+                message: scrub(&message),
+                status: scrub_status(status),
+            },
+            Self::Timeout { message, status } => Self::Timeout {
+                message: scrub(&message),
+                status: scrub_status(status),
+            },
+            Self::Cancelled { message, status } => Self::Cancelled {
+                message: scrub(&message),
+                status: scrub_status(status),
+            },
+            Self::Unavailable { message, status } => Self::Unavailable {
+                message: scrub(&message),
+                status: scrub_status(status),
+            },
+            Self::Status(status) => Self::Status(scrub_status(status)),
+            Self::ProtocolStatus {
+                operation,
+                code,
+                message,
+            } => Self::ProtocolStatus {
+                operation,
+                code,
+                message: scrub(&message),
+            },
+            Self::MalformedReply { detail } => Self::MalformedReply {
+                detail: scrub(&detail),
+            },
+            Self::InvalidArgument { name, detail } => Self::InvalidArgument {
+                name,
+                detail: scrub(&detail),
+            },
+            Self::MxAccess(error) => Self::MxAccess(Box::new(
+                error.with_secrets(secrets.iter().map(|secret| (*secret).to_owned()).collect()),
+            )),
+            other => other,
+        }
+    }
+}
+
 impl From<tonic::Status> for Error {
     fn from(status: tonic::Status) -> Self {
         let message = redact_credentials(status.message());
@@ -461,6 +523,14 @@ fn redact_credentials(message: &str) -> String {
 
 #[cfg(test)]
 mod tests {
+    #[test]
+    fn redact_secrets_scrubs_status_display_and_debug() {
+        let error: super::Error = tonic::Status::internal("boom pw-secret").into();
+        let error = error.redact_secrets(&["pw-secret", ""]);
+        assert!(!format!("{error:?}").contains("pw-secret"));
+        assert!(!error.to_string().contains("pw-secret"));
+    }
+
     use tonic::{Code, Status};
 
     use super::{ensure_mxaccess_success, Error};

clients/rust/tests/client_behavior.rs

@@ -229,6 +229,8 @@ async fn acknowledge_alarm_returns_reply_with_native_status() {
             alarm_full_reference: "Tank01.Level.HiHi".to_owned(),
             comment: "investigating".to_owned(),
             operator_user: "alice".to_owned(),
+            operator_password: String::new(),
+            operator_domain: String::new(),
         })
         .await
         .unwrap();
@@ -1036,6 +1038,30 @@ async fn stream_alarms_emits_snapshot_then_complete_then_transition_in_order() {
     ));
 }
 
+#[tokio::test]
+async fn acknowledge_alarm_error_does_not_leak_operator_password() {
+    let state = Arc::new(FakeState::default());
+    let endpoint = spawn_fake_gateway(state.clone()).await;
+    let client = GatewayClient::connect(ClientOptions::new(endpoint))
+        .await
+        .unwrap();
+
+    let err = client
+        .acknowledge_alarm(AcknowledgeAlarmRequest {
+            client_correlation_id: "corr-pw".to_owned(),
+            alarm_full_reference: "Tank01.Level.HiHi".to_owned(),
+            comment: String::new(),
+            operator_user: "alice".to_owned(),
+            operator_password: "pw-secret".to_owned(),
+            operator_domain: String::new(),
+        })
+        .await
+        .unwrap_err();
+
+    assert!(!format!("{err:?}").contains("pw-secret"));
+    assert!(!err.to_string().contains("pw-secret"));
+}
+
 #[tokio::test]
 async fn cli_subcommands_propagate_unique_correlation_ids_from_next_correlation_id() {
     // The CLI's `stream-alarms` and `acknowledge-alarm` paths used to
@@ -1073,6 +1099,8 @@ async fn cli_subcommands_propagate_unique_correlation_ids_from_next_correlation_
             alarm_full_reference: "Tank01.Level.HiHi".to_owned(),
             comment: String::new(),
             operator_user: String::new(),
+            operator_password: String::new(),
+            operator_domain: String::new(),
         })
         .await
         .unwrap();
@@ -1424,8 +1452,11 @@ impl MxAccessGateway for FakeGateway {
         &self,
         request: Request<AcknowledgeAlarmRequest>,
     ) -> Result<Response<AcknowledgeAlarmReply>, Status> {
-        *self.state.last_correlation_id.lock().await =
+        let request = request.into_inner();
-            Some(request.into_inner().client_correlation_id);
+        if request.operator_password == "pw-secret" {
+            return Err(Status::internal("boom pw-secret"));
+        }
+        *self.state.last_correlation_id.lock().await = Some(request.client_correlation_id);
         Ok(Response::new(AcknowledgeAlarmReply {
             correlation_id: "corr-1".to_owned(),
             protocol_status: Some(ok_status("ack ok")),

docs/AlarmClientDiscovery.md

@@ -621,8 +621,9 @@ returned to normal but is unacknowledged — i.e., visible in the
 "current alarms" list because operator hasn't acked it yet) and
 `UNACK_ALM` (the alarm is currently active and unacknowledged).
 The other states from `eAlmState` (`ACK_RTN`, `ACK_ALM`) would
-appear when an ack is performed — `wwAlarmConsumerClass.AlarmAckByGUID`
+appear when an ack is performed. The alarm-client ack methods
-is the method to call.
+(`AlarmAckByGUID` and friends) do not produce them headless; an `AckMsg`
+write does (`AlarmProbeFindings.md`).
 
 ### `GetStatistics` AV — unrelated quirk
 
@@ -660,7 +661,9 @@ alarm-consumer surface unblocks A.2 fully. Outline:
      events for added/changed/removed records.
    - `AlarmAckByGUID(VBGUID, comment, oprName, node, domain,
      fullName)` for client-driven acknowledgements (matches
-     PR A.5's `AlarmAckCommand` payload).
+     PR A.5's `AlarmAckCommand` payload). *Ack step superseded
+     2026-10-19: acks write `AckMsg` as the authenticated operator
+     (see "Acknowledge in subtag mode" and `AlarmProbeFindings.md`).*
    - Lifecycle teardown: `DeregisterConsumer` +
      `UninitializeConsumer` + `Marshal.FinalReleaseComObject`.
 3. **Conversion layer:** map XML record fields to
@@ -701,8 +704,9 @@ alarm-consumer surface unblocks A.2 fully. Outline:
   `bRetainHiddenAlarms` parameter at all.
 - Subscription expression format: `\\<machine>\Galaxy!<area>`
   (literal `Galaxy` provider) for both libraries.
-- Native ack: `AlarmAckByGUID(VBGUID guid, comment, oprName,
+- Native ack surface (not used by the gateway; superseded 2026-10-19 by
-  node, domain, fullName)` on the v2 surface; ID 5-arg
+  the authenticated `AckMsg` write): `AlarmAckByGUID(VBGUID guid,
+  comment, oprName, node, domain, fullName)` on the v2 surface; ID 5-arg
   variant on the legacy `IwwAlarmConsumer` interface.
 
 These findings retire the open follow-up probes from the
@@ -735,25 +739,32 @@ for `SetXmlAlarmQuery`.
 the canonical `\\<machine>\Galaxy!<area>` form into the XML's
 NODE/PROVIDER/GROUP fields. Mangled or not, the call enables reads.
 
-### 2. Two consumers required: read-side vs. ack-side
+### 2. Two consumers required: read-side vs. ack-side (historical)
+
+> Superseded 2026-10-19: acks now write `AckMsg` as the authenticated operator;
+> see `AlarmProbeFindings.md`. The worker no longer provisions an ack-only
+> wnwrap consumer. The text below records why the split existed.
 
 `SetXmlAlarmQuery` enables reads but **breaks `AlarmAckByName` on
 the same consumer instance**. With SetXml applied, AlarmAckByName
 returns -55 even with valid name+provider+group+operator. Without
 SetXml, AlarmAckByName succeeds with rc=0.
 
-The production consumer therefore provisions **two** wnwrap COM
+The consumer at the time provisioned **two** wnwrap COM
 instances:
 - Primary consumer (`client`): runs full lifecycle including
   `SetXmlAlarmQuery` for `GetXmlCurrentAlarms2` polls.
 - Ack-only consumer (`ackClient`): runs Initialize → Register →
   Subscribe via the v1-prefixed methods, **no SetXmlAlarmQuery**.
-  All `AcknowledgeByName` calls dispatch through this instance.
+  All by-name ack calls dispatched through this instance.
 
 Both consumers subscribe to the same expression. Disposal cleans up
 both via a shared `ReleaseConsumerCom` helper.
 
-### 3. `AlarmAckByName` v2 8-arg vs. v1 6-arg
+### 3. `AlarmAckByName` v2 8-arg vs. v1 6-arg (historical)
+
+> Superseded 2026-10-19: the gateway no longer calls `AlarmAckByName`; see
+> `AlarmProbeFindings.md`.
 
 `wwAlarmConsumerClass` exposes two `AlarmAckByName` overloads:
 - `IwwAlarmConsumer2` v2: 8 args (`name, provider, group, comment,
@@ -762,8 +773,8 @@ both via a shared `ReleaseConsumerCom` helper.
 
 The v2 8-arg method returns -55 on this AVEVA build regardless of
 operator-identity inputs — looks like a stub. The v1 6-arg method
-works. Production `WnWrapAlarmConsumer.AcknowledgeByName` calls the
+works. `WnWrapAlarmConsumer.AcknowledgeByName` called the
-6-arg overload and discards the proto's `domain` + `full_name` fields.
+6-arg overload and discarded the proto's `domain` + `full_name` fields.
 
 **Correction, 2026-08-18** (`docs/AlarmProbeFindings.md`). Both halves
 of the paragraph above are narrower than they read. The -55 tracks the
@@ -775,42 +786,34 @@ snapshot stay `UNACK_ALM`, `OPERATOR_NAME` stay empty, and the alarm
 extension's own `.Acked` attribute stay `False` for 16 s after each
 call. On this rig the wnwrap ack is accepted and then inert, so a zero
 return code must not be read as an applied acknowledgement.
-The proto contract keeps the 8 fields for forward compatibility if
-AVEVA fixes the v2 method later.
 
-### 4. `AlarmAckByGUID` is not implemented
+### 4. `AlarmAckByGUID` is not implemented (historical)
+
+> Superseded 2026-10-19: acks now write `AckMsg` as the authenticated operator;
+> see `AlarmProbeFindings.md`.
 
 The v2 `AlarmAckByGUID(VBGUID, …)` throws `NotImplementedException`
 (COM `E_NOTIMPL`) on `wwAlarmConsumerClass` against this AVEVA
 build. The reference→GUID lookup that we initially planned to wire
-through `AlarmAckByGUID` is therefore not viable on wnwrap; all acks
+through `AlarmAckByGUID` is therefore not viable on wnwrap.
-must go through `AlarmAckByName`.
 
-The proto `AcknowledgeAlarmCommand` (GUID-based) and the worker's
+#### Managed GUID ack experiment — did not work (live-tested 2026-10-18, removed 2026-10-19)
-`MxAccessCommandExecutor.ExecuteAcknowledgeAlarm` switch arm remain
-in the codebase for the forward-compat shape, but the gateway-side
-`WorkerAlarmRpcDispatcher.AcknowledgeAsync` now always routes through
-`AcknowledgeAlarmByName` when the public RPC supplies a recognizable
-`Provider!Group.Tag` reference.
 
-**Opt-in managed GUID ack — does not work on this rig (live-tested
+An opt-in experiment drove `aaAlarmManagedClient.AlarmClient` by reflection
-2026-10-18).** Setting `MXGATEWAY_USE_MANAGED_ALARM_ACK=1` in the
+(its members are non-public) to take GUID-keyed acks in place of the wnwrap
-gateway's environment (workers inherit it) makes the worker build an
+method. The live test showed it could not work, and the experiment and its
-`AaManagedAlarmAckConsumer`, which drives
+enabling flag were removed once the `AckMsg` write replaced it. The findings
-`aaAlarmManagedClient.AlarmClient` by reflection (its members are
+are kept because they explain why the alarm-client route was abandoned.
-non-public) and is meant to take GUID-keyed acks in place of the
-wnwrap method. The live test shows it cannot do that as built, so leave
-the flag off.
 
 The test raised `TestMachine_001.TestAlarm001` with a secured write,
 read its GUID from the wnwrap XML snapshot (the public API exposes no
 alarm GUIDs), and acknowledged it through the `AcknowledgeAlarm` RPC
-with the flag on:
+with the experiment enabled:
 
 | Step | Result |
 |---|---|
-| RPC ack by GUID, flag off (control) | `NotImplementedException`; `.Acked` stays `False` |
+| RPC ack by GUID, experiment off (control) | `NotImplementedException`; `.Acked` stays `False` |
-| RPC ack by GUID, flag on | Same `NotImplementedException`; `.Acked` `False`, `.AckMsg` empty |
+| RPC ack by GUID, experiment on | Same `NotImplementedException`; `.Acked` `False`, `.AckMsg` empty |
 | `AlarmClient.RegisterConsumer` in a process with a registered wnwrap consumer | `-65` — and wnwrap gets `-65` if the managed client registered first; a real `hWnd` does not help |
 | Managed client alone in its own process, records loaded | `AlarmAckByGUID` → `-1`, `SelectByGUID` → `-1` for the wnwrap GUID in both byte orders, `AlarmAckSelected` → `-57` |
 
@@ -820,7 +823,7 @@ Three separate problems follow:
    a wnwrap reader, so the managed `RegisterConsumer` fails with `-65`.
    The dispatcher is built to drop back to the wnwrap ack path on a
    lifecycle failure (so alarm reads keep working), and it does so
-   silently — which is why the flag-on RPC is indistinguishable from the
+   silently — which is why the experiment-on RPC was indistinguishable from the
    control.
 2. **The managed client does not recognise the wnwrap GUID.** `-1` from
    `SelectByGUID` is "not found", not a security refusal, even with the
@@ -833,14 +836,8 @@ Three separate problems follow:
    strings, so the domain and full name the managed method carries
    would not arrive through the public RPC even if 1 and 2 were solved.
 
-A working managed ack would need the client hosted out of process, a way
+The route was dropped in favour of the authenticated `AckMsg` write
-to resolve the alarm to the identifier the managed client uses, and the
+(`AlarmProbeFindings.md`, "Fourth attempt").
-identity fields plumbed through the gateway. Routing rules the worker
-already applies stay correct and would carry over: subtag-mode acks
-(synthetic GUIDs) stay on the subtag consumer, and by-name acks stay on
-wnwrap because `AlarmClient` has no by-name ack. The skip-gated
-`AlarmManagedGuidAckLiveSmokeTests` exercises the in-process
-arrangement and hits problem 1.
 
 ### 5. STA / threading — resolved
 
@@ -996,20 +993,24 @@ AVEVA-internal GUID.
 
 ### Acknowledge in subtag mode
 
-`AlarmDispatcher` routes ack calls by active provider mode:
+Both provider modes acknowledge through the worker's `AlarmAckWriter`, which
+calls `AuthenticateUser` as the requesting operator and writes the comment to
+the ack item over MXAccess on its own `LMXProxyServer`:
 
-- **Alarm-manager mode:** `AlarmAckByName` on `wwAlarmConsumerClass` (unchanged).
+- **Alarm-manager mode:** the item is `<TagName>.AckMsg` for Galaxy-provider
-- **Subtag mode:** `SubtagAlarmConsumer.AcknowledgeByName` resolves the
+  alarms. An alarm from another provider is not ackable.
-  watch-list entry's `ack_comment_subtag` and issues a `Write(comment)` on
+- **Subtag mode:** `SubtagAlarmConsumer` resolves the watch-list entry's
-  the STA via `LmxSubtagAlarmSource`. Writing the `AckMsg` subtag performs
+  `ack_comment_subtag` and the writer writes that item verbatim. Writing the
-  the acknowledge in AVEVA (`AckMsg` is the confirmed `AlarmExtension` ack-comment
+  `AckMsg` subtag performs the acknowledge in AVEVA (`AckMsg` is the confirmed
-  write target).
+  `AlarmExtension` ack-comment write target).
 
-If the alarm has no writable ack-comment subtag (`AckComment` config key is
+Because the write is authenticated, the Historian records the operator's name,
-empty, or the entry's `ack_comment_subtag` field is empty), the ack call
+domain and full name, and the writer's MXAccess client name
-returns a failure code that the gateway surfaces as `FailedPrecondition`.
+(`MxGateway:Alarms:AckClientName`) as OperatorNode. If the alarm has no ack
-`AcknowledgeByGuid` maps the synthetic GUID back to its reference via an
+target (`AckComment` config key is empty, the entry's `ack_comment_subtag` is
-internal dictionary, then calls the same write path.
+empty, or the reference or synthetic GUID is unknown), the worker reports it
+as not found or not ackable and the gateway surfaces `InvalidRequest` with
+`hresult` -1.
 
 `SubtagAlarmConsumer.Subscribe` advises the ack-comment subtag alongside the
 observed ones (active/acked/priority). This is required: MXAccess rejects a
@@ -1027,7 +1028,7 @@ The subtag path was validated against live MXAccess on the dev rig
   `…​.Priority` → `500` (Int32), `…​.AckMsg` → string — confirming the field
   names **and** the runtime reference shape `<Object>.<AlarmAttr>.<field>`
   with **no** intermediate alarm-condition segment.
-- `AcknowledgeByName` (AckMsg write) returned `0` once the ack-comment subtag
+- The `AckMsg` write returned `0` once the ack-comment subtag
   was advised — confirming the ack-by-comment-write mechanism end to end.
 
 ### Fidelity limitations
@@ -1050,6 +1051,18 @@ Every transition and snapshot record carries `degraded = true` and
 `source_provider = ALARM_PROVIDER_MODE_SUBTAG`. Clients that require full
 fidelity must wait for failback to the alarm manager.
 
+### Retrigger transitions
+
+`ALARM_TRANSITION_KIND_RETRIGGER` is reserved in the contract and the worker
+never emits it, in either provider mode. The alarmmgr XML reports only the
+four states `UNACK_ALM`, `ACK_ALM`, `UNACK_RTN`, and `ACK_RTN`, so a condition
+that returns to alarm after clearing (`*_RTN` → `*_ALM`) looks the same as a
+new raise, and `AlarmRecordTransitionMapper.MapTransition` reports it as
+`ALARM_TRANSITION_KIND_RAISE`. The subtag source has no re-alarm counter to
+observe either. The worker does not infer a retrigger from timing or
+from a raise that follows a clear, because the gateway forwards only
+transitions its source reports.
+
 ### Provider mode reflection
 
 When `FailoverAlarmConsumer` switches between providers, it raises

docs/AlarmProbeFindings.md

@@ -24,13 +24,15 @@ the blocker rather than rediscovering it.
 |---|---|
 | GUID stable across polls and `ALM → RTN` | Answered — yes (2026-05-01 capture in `AlarmClientDiscovery.md`, reconfirmed 2026-08-18) |
 | GUID stable across clear-then-re-raise | Answered 2026-08-18 — **no**, a re-raise mints a new GUID |
-| GUID stable across `UNACK → ACK` | **Open** — the rig cannot be driven into an acknowledged state; 2026-08-18 (third attempt) identifies the reason as the test attribute's `MxSecurityOperate` classification, which no non-interactive ack surface on the rig can satisfy |
+| GUID stable across `UNACK → ACK` | Blocker resolved 2026-10-19 — the rig is driven to `ACK_ALM` by an `AckMsg` write, and the `MxSecurityOperate` classification was not the cause (see "Fourth attempt"); GUID survival across that transition has not been recorded yet |
+| Acknowledge an alarm headless | Answered 2026-10-19 — **an `AckMsg` write as the authenticated operator**; every alarm-client ack is inert (see "Fourth attempt") |
 | `COUNT` = total active vs records-in-reply under a capped fetch | Answered 2026-08-18 — **records in the reply** |
 
 The 2026-08-17 run below is kept because it is the record of the wrong-verb blocker. The
 2026-08-18 run cleared that blocker with `AuthenticateUser` + `WriteSecured` and answered
 two of the three questions; the acknowledge leg is now blocked on something narrower and
-different, described in "Second attempt" and diagnosed in "Third attempt".
+different, described in "Second attempt" and diagnosed in "Third attempt"
+(resolved in the Fourth attempt, 2026-10-19).
 
 ## First attempt (2026-08-17): plain `Write`
 
@@ -420,3 +422,126 @@ The second attempt's inventory, unchanged since: the three
 snapshot, since nothing can acknowledge them away. `SnapshotActiveAlarms` counts only
 `UNACK_ALM` and `ACK_ALM` as active, so these are inert for the gateway; they will clear on
 the next `alarmmgr` restart.
+
+## Fourth attempt (2026-10-19): authenticated AckMsg write
+
+The third attempt left the acknowledge leg blocked. This run found a route that
+acknowledges and identifies the operator, and ruled out the alarm-client routes.
+It drove `TestMachine_001.TestAlarm001`, raised with
+`mxa write TestMachine_001.TestAlarm001 true --type bool --secured -u Administrator`.
+
+### Writing `AckMsg` acknowledges
+
+Writing the comment to `TestMachine_001.TestAlarm001.AckMsg` over MXAccess
+acknowledges the alarm: `.Acked` becomes `True` and the wnwrap record moves to
+`STATE=ACK_ALM` with an `ACK_COMMENT`. `AckMsg` is `MxSecurityFreeAccess` even
+though the alarm UDA itself is `MxSecurityOperate`, so no Galaxy change is
+needed.
+
+What the Historian records, read from `Runtime.dbo.v_AlarmEventHistory2`:
+
+| Write | Operator | DomainName | UserFullName | OperatorNode | Description |
+|---|---|---|---|---|---|
+| Unauthenticated `Write` | `NULL` | `NULL` | `NULL` | `NULL` | the comment |
+| `AuthenticateUser` + `Write(value, userId)` | `Administrator` | `ArchestrA` | `Administrator` | `mxa` (the MXAccess `Register()` client name) | the comment |
+
+The wnwrap XML also gains `ACK_OPERATOR_NAME`, `ACK_DOMAIN_NAME`,
+`ACK_OPERATOR_NODE` and `ACK_COMMENT` on the acknowledged record; the worker's
+snapshot parser reads them for Acknowledge transitions. An unauthenticated write
+therefore acknowledges but leaves no operator in the Historian, which is why the
+gateway requires `operator_user` and authenticates it.
+
+Another actor's `AckMsg` writes on the same day (authenticated as another
+operator account) logged that account as `Operator`, `Domain="."` and that machine's name as the node,
+which matches the client-name rule. By this date unauthenticated writes to
+`MxSecurityOperate` attributes were also accepted Galaxy-wide on the rig, where
+August refused them with 1008.
+
+### Alarm-client acks are inert in every variant
+
+All of these returned `rc=0` or `-1` and changed neither the snapshot, `.Acked`
+nor the Historian:
+
+- wnwrap `AlarmAckByName` and `AlarmAckByGUID` (`E_NOTIMPL`) on the ack-only and
+  reader consumers.
+- wnwrap `IwwAlarmConsumer2` on a sole consumer with `SetXmlAlarmQuery` applied
+  (list populated, `GetAlarm(0)` returns the row): `AlarmAck` with ids
+  0/1/2/3/-1/65536, and `Select(0)` + `AlarmAckSelected`, with operators
+  `TestOperator`, `Administrator` and a third account. `SelectByGUID` is `E_NOTIMPL`.
+- The same calls with `TestAlarm001` temporarily reclassified to
+  `MxSecurityFreeAccess` (template and instance, restored afterwards), so the
+  security classification is not the cause.
+- Managed `aaAlarmManagedClient`: `AlarmAckByGUID` and `SelectByGUID` return
+  `-1` (the client's list never fills headless, even with a real `hWnd` and a
+  message pump); `AckSelectedTagAlarms_Ex` returns `0` even for a
+  `NOSUCHPROVIDER` provider and writes no Historian row. The managed and wnwrap
+  consumers cannot share a process (`RegisterConsumer` returns `-65`).
+- A 20 s continuous message pump after `AlarmAck`, and registering as `InTouch`,
+  `WindowViewer` or `Wonderware InTouch`.
+
+Static reverse engineering matched: the five-string COM acks forward domain and
+full name with an empty password, and InTouch's `AlarmViewer.ocx` also sends an
+empty password. The user-request call is synchronous request-to-return-code with
+no completion channel, so the drop happens provider-side. Hooking the user
+request in a live WindowViewer that acknowledges successfully and diffing it
+against the headless probe is the one definitive step not taken.
+
+### Consequence
+
+The gateway acknowledges by authenticating the operator and writing `AckMsg`
+(`docs/DesignDecisions.md`, "Alarm acknowledgement writes AckMsg as the
+authenticated operator"). The managed ack consumer and the second wnwrap ack
+consumer were removed.
+
+### Gateway-level verification
+
+The same result holds end to end through the gateway. With the feature build deployed on
+the windev rig and `TestAlarm001` raised, an `AcknowledgeAlarm` RPC for
+`Galaxy!TestArea.TestMachine_001.TestAlarm001` with `operator_user=Administrator` and an
+empty password returned `PROTOCOL_STATUS_CODE_OK`, `hresult` 0. `.Acked` turned True,
+`.AckMsg` held the comment, and the Historian logged the `ACK_ALM` row with
+`Operator=Administrator`, `DomainName=ArchestrA`, `UserFullName=Administrator`,
+`OperatorNode=MxGateway` (the default `AckClientName`), and `Description` = the comment.
+
+A wrong password, or a user Galaxy security does not know, makes `AuthenticateUser`
+throw `ArgumentException` (`E_INVALIDARG`, 0x80070057) rather than a `COMException`.
+That is why the writer treats `ArgumentException` from the MXAccess call itself as an
+interop failure alongside `COMException`, and reports the HRESULT the exception carries.
+Such an ack returns `PROTOCOL_STATUS_CODE_MXACCESS_FAILURE`, the alarm stays unacked,
+the Historian gains no ack row, and the password string does not appear in the gateway
+log, `stdout.log`, or `stderr.log`. A non-Galaxy provider reference returns
+`PROTOCOL_STATUS_CODE_INVALID_REQUEST` with `hresult` -1, and an empty `operator_user`
+returns gRPC `InvalidArgument`.
+
+The GUID arm of the ack command was not driven live. The public alarm RPCs expose only
+the full reference, so clients reach the by-name arm; worker unit tests cover the GUID
+arm.
+
+## Item handle reuse
+
+`AlarmAckWriter` correlates an ack with its `OnWriteComplete` by the
+(server handle, item handle) pair, because the callback carries no per-write id.
+After an unconfirmed ack it drops the `AckMsg` item and re-adds it so the next
+ack waits on a new handle that a late completion for the old one cannot match.
+That only works if MXAccess does not hand the same number out again.
+
+`ItemHandleReuseProbeTests` (Skip-gated, worker tests) checked this on windev
+(`10.100.0.48`, Galaxy `DEV`) on 2026-10-20 against
+`TestMachine_001.TestAlarm001.AckMsg`, without writing:
+
+| Sequence on one registration (server handle 1) | Item handles returned |
+|---|---|
+| `AddItem` + `Advise`, then `UnAdvise` + `RemoveItem`, five times | 1, 2, 3, 4, 5 |
+| `AddItem`, then `RemoveItem`, five times | 6, 7, 8, 9, 10 |
+| Add and remove `AckMsg`, add `Acked`, add `AckMsg` again | 11, then 12 (`Acked`), 13 |
+| Add `AckMsg` twice while both are held | 14, 15 |
+
+Within a registration the numbers increase and a freed number is never reused,
+so the drop-and-re-add mitigation holds. Numbering is per registration:
+`Unregister` + `Register` on the same COM object returned server handle 2 and
+item handle 1, and a new `LMXProxyServerClass` returned server handle 1 and
+item handle 1 again. The writer is not exposed to that restart because its
+teardown detaches its `OnWriteComplete` handler from the old COM object and
+clears that server handle's completion entries before it creates a new one.
+No per-handle generation counter is needed.
+

docs/Authorization.md

@@ -125,7 +125,7 @@ public string ResolveRequiredScope(object request)
 }
 ```
 
-The `_ => GatewayScopes.Admin` fallback is intentional: any future request type that the resolver does not recognize fails closed, requiring the strongest scope until the resolver is updated. `AcknowledgeAlarm` is treated as a write — it mutates alarm state, mirroring `MxCommandKind.Write*` — and `StreamAlarms` and `QueryActiveAlarms` share the alarm/event surface with `StreamEvents` and `MxCommandKind.DrainEvents`, so they carry `events:read` (the active-alarm snapshot is the same data reachable through the event surface). All three alarm RPCs are session-less: the scope check is the only authorization gate, since there is no per-session ownership to enforce.
+The `_ => GatewayScopes.Admin` fallback is intentional: any future request type that the resolver does not recognize fails closed, requiring the strongest scope until the resolver is updated. `AcknowledgeAlarm` is treated as a write — it mutates alarm state, mirroring `MxCommandKind.Write*`. It stays `invoke:write` rather than `invoke:secure` because the request carries the operator's own Galaxy credential, which the worker authenticates before the `AckMsg` write; the credential, not the API-key scope, authorizes the identity the Historian records. The trade-off is that an `invoke:write` key can tell `AuthenticationFailed` (an `MXACCESS_FAILURE` whose diagnostic reports operator authentication failure) apart from every other outcome, which makes `AcknowledgeAlarm` a Galaxy-password oracle for that key — whereas the `AuthenticateUser` command itself requires `invoke:secure`. What bounds it: the caller must already hold a valid, admin-issued, revocable `invoke:write` key, and every acknowledgement is serialized through the single monitor worker session's STA, so guesses run one at a time. The gateway does **not** throttle failed operator authentications: the `ApiKeyFailureLimiter` counts only failed API-key verifications, and the ASP.NET Core rate limiter covers only the dashboard `/login` endpoint. Any lockout after repeated bad passwords comes from the Galaxy security configuration, not from the gateway. Grant `invoke:write` only to clients trusted with that exposure. `StreamAlarms` and `QueryActiveAlarms` share the alarm/event surface with `StreamEvents` and `MxCommandKind.DrainEvents`, so they carry `events:read` (the active-alarm snapshot is the same data reachable through the event surface). All three alarm RPCs are session-less: the scope check is the only authorization gate, since there is no per-session ownership to enforce.
 
 `MxCommandRequest` is special because it multiplexes many MxAccess operations through a single RPC. The resolver inspects the embedded `MxCommandKind` so each operation gets its own scope:
 

docs/Contracts.md

@@ -49,6 +49,17 @@ worker prepares the replacement on its STA before swapping it into the existing
 failover consumer and returns a normal `MxCommandReply`; callers must treat only
 `protocol_status = OK` as acknowledgement of the new list.
 
+`AcknowledgeAlarmRequest` carries the operator's identity: `operator_user`
+(required), `operator_password` (field 6) and `operator_domain` (field 7). The
+gateway copies them onto `AcknowledgeAlarmCommand` (`operator_password` = 7) and
+`AcknowledgeAlarmByNameCommand` (`operator_password` = 9); the worker
+authenticates the operator and writes `AckMsg` as that user. The fields are
+additive, so `GatewayProtocolVersion` stays `3`. `operator_password` is a
+credential and follows the same never-log rule as
+`AuthenticateUserCommand.verify_user_password`: it must not reach logs, metric
+labels, `DiagnosticMessage`, `ProtocolStatus.Message`, or exception text. A
+not-found or not-ackable alarm is reported with `hresult` -1.
+
 The command model also includes bulk write/read command kinds:
 `WriteBulk`, `Write2Bulk`, `WriteSecuredBulk`, `WriteSecured2Bulk`, and
 `ReadBulk`. They are unary `Invoke` payloads on the same `MxAccessGateway`

docs/DesignDecisions.md

@@ -703,14 +703,15 @@ latch in subtag mode.
 
 ### Acknowledge via ack-comment write
 
-In subtag mode, `AcknowledgeAlarm` writes the operator comment to the alarm
+Both provider modes acknowledge by writing the operator comment to `AckMsg` as
-attribute's ack-comment subtag (`Fallback:Subtags:AckComment`). The write
+the authenticated operator (see "Alarm acknowledgement writes AckMsg as the
-performs the native ack in AVEVA. This differs from alarmmgr mode, where
+authenticated operator" below). In alarm-manager mode the target is
-`AlarmAckByName` on `wwAlarmConsumerClass` is called directly. The `AckComment`
+`<TagName>.AckMsg`; in subtag mode it is the watch-list entry's ack-comment
-subtag name is empty by default; configuring it is required for ack to work in
+subtag (`Fallback:Subtags:AckComment`, default `AckMsg`). The write performs
-subtag mode. The exact AVEVA subtag names are not hard-coded — the `Subtags`
+the ack in AVEVA. An entry with an empty ack-comment subtag is not ackable and
-config block exists precisely so names are not guessed without validation
+the RPC reports `InvalidRequest`. The exact AVEVA subtag names are not
-against the live MXAccess attribute set.
+hard-coded — the `Subtags` config block exists so names are not guessed
+without validation against the live MXAccess attribute set.
 
 ### Related documentation
 
@@ -784,6 +785,47 @@ The following items were previously listed here and have since shipped:
   `MxGateway:Sessions:MaxEventSubscribersPerSession`.
   See [Event Subscribers](#event-subscribers) above and [Sessions](./Sessions.md).
 
+## Alarm acknowledgement writes AckMsg as the authenticated operator
+
+**Decision (2026-10-19).** `AcknowledgeAlarm` authenticates the requesting
+operator with `AuthenticateUser` (`operator_user`, `operator_password`,
+optional `operator_domain`) and writes the comment to `<TagName>.AckMsg` over
+MXAccess as that user, on a dedicated `LMXProxyServer` registered as
+`MxGateway:Alarms:AckClientName`. Both provider modes use this path.
+
+**Why.** Live tests (2026-10-18/19) showed every alarm-client ack is inert on a
+headless consumer: wnwrap `AlarmAckByName`/`AlarmAckByGUID`/`AlarmAck`/
+`AlarmAckSelected`, and the managed `aaAlarmManagedClient` GUID ack, return
+rc=0 or -1 and change nothing, regardless of security classification,
+operator name or consumer name. An MXAccess write to `AckMsg` does acknowledge
+the alarm, and when the write is made under an authenticated user the AVEVA
+runtime historizes that identity (`Operator`, `DomainName`, `UserFullName`),
+with the MXAccess client name as `OperatorNode`. The requirement is that the
+real operator appears in the Historian, as with InTouch, so a shared gateway
+service account was rejected. Operators have Galaxy accounts, so the
+per-request credential is workable. See
+[Alarm Probe Findings](./AlarmProbeFindings.md).
+
+**Consequences.**
+
+- `operator_user` is required; `operator_password` may be empty.
+- `operator_password` is a credential. It is forwarded to the worker over the
+  pipe but never logged: `AcknowledgeAlarm` and `AcknowledgeAlarmByName` are
+  credential-bearing commands and any property named like `password` is
+  redacted.
+- `Applied` and `Unconfirmed` (write sent, no `OnWriteComplete` in the bounded
+  wait) both report `OK`; clients that need certainty watch for the transition.
+- The managed ack consumer, the WNAL native ack consumer, the second wnwrap
+  ack-only consumer and the opt-in managed-ack environment flag are removed.
+
+**Upgrade note.** This is a breaking change for callers. They must now send
+`operator_user` (required; empty is rejected with `InvalidArgument`) and the
+operator's Galaxy password; a caller that sends no password for a
+password-protected account gets `MXACCESS_FAILURE` from `AuthenticateUser`
+instead of an acknowledgement. OtOpcUa's `GalaxyDriver` alarm acknowledger is a
+known caller that must be updated to forward the operator's credentials before
+the gateway is upgraded.
+
 ## Related Documentation
 
 - [Gateway Process Detailed Design](./GatewayProcessDesign.md)

docs/Diagnostics.md

@@ -4,7 +4,7 @@ The diagnostics subsystem provides structured logging, credential redaction, and
 
 ## Goals
 
-The subsystem exists to satisfy two security rules from `gateway.md`: never log passwords or raw credential values for `AuthenticateUser`, `WriteSecured`, or related secured operations, and never log full MXAccess values by default. Code paths that touch credentials or tag values must therefore route through `GatewayLogRedactor` rather than emitting them directly.
+The subsystem exists to satisfy two security rules from `gateway.md`: never log passwords or raw credential values for `AuthenticateUser`, `WriteSecured`, `AcknowledgeAlarm` (its `operator_password`), or related secured operations, and never log full MXAccess values by default. Code paths that touch credentials or tag values must therefore route through `GatewayLogRedactor` rather than emitting them directly.
 
 A second goal is parity-test diagnosability. Because MXAccess sessions, workers, correlation ids, and command methods are the units of comparison, every log entry produced inside a request scope must carry those identifiers without each call site having to format them.
 
@@ -70,7 +70,9 @@ private static readonly HashSet<string> SensitiveCommandMethods = new(StringComp
 {
     "AuthenticateUser",
     "WriteSecured",
-    "WriteSecured2"
+    "WriteSecured2",
+    "AcknowledgeAlarm",
+    "AcknowledgeAlarmByName"
 };
 
 public static bool IsCredentialBearingCommand(string? commandMethod)
@@ -80,7 +82,7 @@ public static bool IsCredentialBearingCommand(string? commandMethod)
 }
 ```
 
-The names match the MXAccess command list in `gateway.md` exactly. `Write` and `Write2` are not in the set because their payloads are tag values, not credentials, and are governed by the `valueLoggingEnabled` flag described below.
+The names match the MXAccess command list in `gateway.md` exactly. `Write` and `Write2` are not in the set because their payloads are tag values, not credentials, and are governed by the `valueLoggingEnabled` flag described below. `AcknowledgeAlarm` and `AcknowledgeAlarmByName` are in the set because their commands carry the operator's `operator_password`. Independently of the method set, the name-based seam redacts any property whose name contains `password`, so a credential field added to a payload later is redacted without a code change.
 
 ### API key redaction
 

docs/GatewayConfiguration.md

@@ -442,17 +442,18 @@ behavior.
 | `MxGateway:Alarms:ReconcileIntervalSeconds` | `30` | How often the monitor re-resolves and, on material change, hot-swaps the subtag fallback watch-list, then reconciles its in-process alarm cache against the worker's authoritative active-alarm snapshot. The acknowledged watch-list fingerprint advances only after an OK worker reply. Floored at 5 seconds. |
 | `MxGateway:Alarms:PollIntervalMilliseconds` | `500` | Cadence at which the worker's STA polls the AVEVA alarm consumer (`GetXmlCurrentAlarms2`) for the active-alarm snapshot the live feed diffs. Must be between `100` and `3600000` (one hour): every poll is a COM call plus an XML parse on the same STA that serves reads and writes, so a tighter cadence starves the command path, while a value above an hour stops being a cadence and silently disables alarm polling. The gateway conveys the value to the worker via the `MXGATEWAY_ALARM_POLL_INTERVAL_MS` environment variable; a missing or out-of-range value leaves the worker on the 500 ms default rather than failing the session. |
 | `MxGateway:Alarms:MaxAlarmsPerFetch` | `1024` | Cap the worker passes to `GetXmlCurrentAlarms2`'s `maxAlmCnt`. Must be between `64` and `65536` — the worker is a 32-bit process that materializes each reply as one BSTR plus a full `XmlDocument`, so an unbounded cap faults the STA with an out-of-memory rather than merely slowing it. It doubles as the **truncation threshold**: a fetch returning exactly this many records is treated as truncated, because the COM API caps its reply with no "more available" flag. On a truncated poll the worker retains the alarms the capped reply could not mention instead of letting their absence read as a clear, and logs a rate-limited `AlarmSnapshotTruncated` warning to its stderr (identifiers and counts only). **Remediation when you see that warning: raise this value** so the steady-state active-alarm count fits inside one fetch. A galaxy permanently above the cap holds stale entries in the snapshot until a sub-cap poll, and loses clear→re-raise cycles that happen entirely out of window (see `docs/DesignDecisions.md`). Conveyed to the worker via the `MXGATEWAY_ALARM_MAX_ALARMS_PER_FETCH` environment variable; a missing or out-of-range value leaves the worker on the 1024 default. |
+| `MxGateway:Alarms:AckClientName` | `MxGateway` | MXAccess client name the alarm worker's ack writer registers with; the Historian records it as each acknowledgement's OperatorNode. 1-64 printable characters. Conveyed via `MXGATEWAY_ALARM_ACK_CLIENT_NAME`. |
 
 The alarm monitor is independent of client sessions: `AcknowledgeAlarm` and
 `StreamAlarms` are session-less RPCs served by the monitor.
 
-The worker also reads `MXGATEWAY_USE_MANAGED_ALARM_ACK` (inherited from the
+`AckClientName` is the MXAccess client name the alarm worker's ack writer
-gateway process). Setting it to `1` is meant to route GUID-keyed acknowledges
+registers its dedicated `LMXProxyServer` with. The Historian records it as the
-through AVEVA's managed alarm client, but a live test (2026-10-18) showed the
+OperatorNode of each acknowledgement, so it identifies the gateway as the
-path cannot work as built: the managed client fails to register alongside the
+acknowledging node. The gateway passes it to each worker through the
-worker's wnwrap reader and the worker silently falls back to wnwrap. Leave it
+`MXGATEWAY_ALARM_ACK_CLIENT_NAME` environment variable. Startup validation
-unset. It has no `MxGateway:*` option. See
+rejects an empty value, a value over 64 characters, or one containing control
-[AlarmClientDiscovery](./AlarmClientDiscovery.md) ("Opt-in managed GUID ack").
+characters.
 
 ### Alarm fallback options
 
@@ -471,7 +472,7 @@ native wnwrap alarm-manager provider and the subtag-monitoring fallback.
 | `MxGateway:Alarms:Fallback:Discovery:ExcludeAttributes` | _(empty)_ | Attribute paths to remove from the merged watch-list (case-insensitive). The exclude is applied after the Repository-derived rows and the explicit `IncludeAttributes` entries are combined, so an exclude that matches an explicit include suppresses it too — excludes win. Ignored when `UseGalaxyRepository` is `false`. |
 | `MxGateway:Alarms:Fallback:Subtags:Active` | `InAlarm` | Subtag name for the in-alarm boolean. Confirmed AVEVA `AlarmExtension` field name. |
 | `MxGateway:Alarms:Fallback:Subtags:Acked` | `Acked` | Subtag name for the acknowledged boolean. Confirmed AVEVA `AlarmExtension` field name. |
-| `MxGateway:Alarms:Fallback:Subtags:AckComment` | `AckMsg` | Subtag name for the acknowledgement comment write target. Writing this subtag performs the acknowledge in AVEVA. Confirmed AVEVA `AlarmExtension` field name. When empty, the ack-comment write path is disabled. |
+| `MxGateway:Alarms:Fallback:Subtags:AckComment` | `AckMsg` | Subtag name for the acknowledgement comment write target. Writing this subtag, as the authenticated operator, performs the acknowledge in AVEVA. Confirmed AVEVA `AlarmExtension` field name. When empty, the alarm is not ackable and the RPC returns `InvalidRequest`. |
 | `MxGateway:Alarms:Fallback:Subtags:Priority` | `Priority` | Subtag name for the alarm priority / severity value. Confirmed AVEVA `AlarmExtension` field name. |
 
 Validation rules:

docs/Grpc.md

@@ -96,9 +96,30 @@ Selected control commands have additional public behavior beyond their protobuf
 
 ### `AcknowledgeAlarm`
 
-`AcknowledgeAlarm` is a unary, **session-less** RPC that acknowledges a single alarm. The handler validates `alarm_full_reference` inline (it does not run through `MxAccessGrpcRequestValidator`) and delegates to `IGatewayAlarmService.AcknowledgeAsync`. The always-on `GatewayAlarmMonitor` routes the ack over its own gateway-managed worker session — clients no longer open a session to acknowledge an alarm. A reference that parses as a canonical GUID forwards to `AcknowledgeAlarmCommand`; a `Provider!Group.Tag` reference forwards to `AcknowledgeAlarmByNameCommand`.
+`AcknowledgeAlarm` is a unary, **session-less** RPC that acknowledges a single alarm. The handler validates `alarm_full_reference` and `operator_user` inline (it does not run through `MxAccessGrpcRequestValidator`) and delegates to `IGatewayAlarmService.AcknowledgeAsync`. The always-on `GatewayAlarmMonitor` routes the ack over its own gateway-managed worker session — clients no longer open a session to acknowledge an alarm. A reference that parses as a canonical GUID forwards to `AcknowledgeAlarmCommand`; a `Provider!Group.Tag` reference forwards to `AcknowledgeAlarmByNameCommand`.
 
-An `OK` response means the alarm provider accepted the acknowledgement, not that it applied it: the gateway forwards the ack and reports what the provider returned, in keeping with the parity rule. On galaxies whose alarmed attributes carry a non-free-access security classification the by-name ack is accepted and inert — the underlying `AlarmAckByName` conveys an operator name rather than an authenticated identity, observed on the probe rig with the mechanism inferred rather than confirmed — so a client that needs to confirm an acknowledgement should watch for the resulting transition on `StreamAlarms` instead of treating `OK` as proof. See [Alarm Probe Findings](./AlarmProbeFindings.md).
+Request fields that identify the acknowledging operator:
+
+- `operator_user` (required): the Galaxy account name. An empty value is rejected with `InvalidArgument` so the real operator is always recorded.
+- `operator_password` (field 6): the operator's Galaxy password. May be empty because Galaxy accounts can have empty passwords. It is a credential and is never logged.
+- `operator_domain` (field 7): optional. When set, the worker authenticates `domain\user`; when empty it authenticates `user`.
+
+The worker calls `AuthenticateUser` as the operator, then writes `operator_comment` to `<TagName>.AckMsg` (subtag mode: the configured ack-comment subtag) on a dedicated MXAccess server registered as `MxGateway:Alarms:AckClientName`, and waits a bounded time for `OnWriteComplete`. The AVEVA runtime treats that write as the acknowledgement, and the Historian (`Runtime.dbo.v_AlarmEventHistory2`) records the operator name, domain and full name from the authenticated account, with the ack client name as `OperatorNode`. The alarm-client ack calls (`AlarmAckByName`, `AlarmAckByGUID`) are not used because a headless consumer's acks are accepted and dropped; see [Alarm Probe Findings](./AlarmProbeFindings.md).
+
+| Worker outcome | Response status | `hresult` |
+|---|---|---|
+| Write applied, or sent but no `OnWriteComplete` within the wait (unconfirmed, with a diagnostic) | `OK` | `0` |
+| Unknown alarm, or no ack target (non-Galaxy provider, no ack-comment subtag) | `INVALID_REQUEST` | `-1` |
+| `AuthenticateUser` raised an MXAccess error (bad user or wrong password surfaces as `E_INVALIDARG` `0x80070057`, raised by the interop as `ArgumentException`; `COMException` and `UnauthorizedAccessException` are classified the same way) | `MXACCESS_FAILURE` | the exception's HRESULT |
+| `AuthenticateUser` returned user id `0` | `MXACCESS_FAILURE` | `-1` |
+| `AuthenticateUser` failed with a server-gone HRESULT (`RPC_E_DISCONNECTED`, `RPC_S_SERVER_UNAVAILABLE`, `RPC_E_SERVERFAULT`); the writer tears down its MXAccess registration and re-registers on the next ack | `MXACCESS_FAILURE` | that HRESULT |
+| `AddItem`, `Advise` or `Write` on the `AckMsg` item raised an MXAccess error (`COMException`, or `ArgumentException` for `E_INVALIDARG`); the item is dropped and re-added on the next ack, and a server-gone HRESULT also tears down the registration | `MXACCESS_FAILURE` | the exception's HRESULT |
+| Write rejected (`OnWriteComplete` non-OK) | `MXACCESS_FAILURE` | the `detail` of the `OnWriteComplete` `MXSTATUS_PROXY`, or `-1` when `detail` is `0` |
+| Any other exception escaping the ack (e.g. the ack writer's MXAccess `Register` failing, the monitor's worker session having no active alarm subscription — `InvalidOperationException` — or an exception from the worker's own code rather than from an MXAccess call: interop failures are classified only when the `AuthenticateUser`, `AddItem`, `Advise` or `Write` call itself raises them, and an `ArgumentNullException` or `ArgumentOutOfRangeException` is never classified because the interop does not raise them for an HRESULT) | `MXACCESS_FAILURE` | the exception's `HResult`; the diagnostic names only the exception type and HRESULT |
+
+`reply.Statuses` carries the `OnWriteComplete` `MxStatusProxy` rows when they were observed. Because an unconfirmed write also returns `OK`, a client that must be certain should watch for the resulting transition on `StreamAlarms`. `AcknowledgeAlarm` stays under the `invoke:write` scope; see [Authorization](./Authorization.md).
+
+**Upgrade note.** Callers must now send `operator_user` (required) and the operator's Galaxy password in `operator_password`. A caller that sends no `operator_user` gets `InvalidArgument`; a caller that sends no password for a password-protected Galaxy account gets `MXACCESS_FAILURE` from `AuthenticateUser` instead of an acknowledgement. Every existing caller must be updated before the gateway is upgraded — known callers include OtOpcUa's `GalaxyDriver` alarm acknowledger, which must supply the operator's credentials.
 
 ### `StreamAlarms`
 
@@ -217,6 +238,7 @@ receive this event directly.
 | `StreamEvents` | `session_id` must be non-empty. | `InvalidArgument` |
 | `Invoke` | `session_id` non-empty, `command` present, `kind` not `Unspecified`, payload oneof must match `kind`. | `InvalidArgument` |
 | `AcknowledgeAlarm` | `alarm_full_reference` must be non-empty. Validated inline in the handler, not by `MxAccessGrpcRequestValidator`. | `InvalidArgument` |
+| `AcknowledgeAlarm` | `operator_user` must be non-empty. Validated inline in the handler. | `InvalidArgument` |
 | `StreamAlarms` | No required fields — `alarm_filter_prefix` is optional. | — |
 | `QueryActiveAlarms` | No required fields — `alarm_filter_prefix` is optional. Not routed through `MxAccessGrpcRequestValidator`; the handler only null-checks the request. | — |
 

docs/MxAccessWorkerInstanceDesign.md

@@ -133,6 +133,9 @@ starts the STA, or touches MXAccess:
 | `2` | `InvalidArguments` | Required arguments are missing or unknown arguments are present. |
 | `3` | `InvalidProtocolVersion` | `--protocol-version` is not numeric or does not match the supported worker protocol. |
 | `4` | `MissingNonce` | `MXGATEWAY_WORKER_NONCE` is absent or empty. |
+| `5` | `PipeConnectionFailed` | An `IOException` or `TimeoutException` escapes the pipe client. |
+| `6` | `ProtocolViolation` | A `WorkerFrameProtocolException` escapes the pipe client. |
+| `7` | `UnhandledException` | An exception reaches the `AppDomain.UnhandledException` hook (see `docs/WorkerBootstrap.md`). |
 
 Bootstrap logs use `WorkerConsoleLogger` key/value output. `WorkerLogRedactor`
 redacts fields whose names indicate nonce, secret, password, token,
@@ -270,12 +273,33 @@ is still responsive. Shutdown marks the runtime as closing, wakes the pump,
 rejects new commands, cancels queued work, uninitializes COM on the STA, and
 waits for the thread to exit.
 
+`ProcessQueuedCommands` pumps pending Windows messages between queued work
+items whenever more work remains, not only after the whole queue drains. A
+burst of queued commands would otherwise hold every inbound COM callback
+(`OnDataChange`, `OnWriteComplete`) behind the entire backlog.
+
+If the STA loop itself throws after `Start` has returned (a failed
+`MsgWaitForMultipleObjectsEx`, or an exception escaping a dispatched message),
+`StaRuntime` records the exception as `LoopFault`, fails every later
+`InvokeAsync` with an `InvalidOperationException` wrapping it (never the
+graceful `StaRuntimeShutdownException`), cancels queued work, and invokes the
+handler registered through `RegisterLoopFaultHandler` — unless a requested
+shutdown is already in progress, in which case the death is part of a
+deliberate teardown and is not reported. `MxAccessStaSession` registers a
+handler that records a `StaHung` `WorkerFault` on the event queue (diagnostic
+text carries the exception type and HRESULT only, never the message),
+so the IPC drain loop sends the fault to the gateway and ends the pipe session
+and the worker process. Before this, a post-start loop death was stored in the
+startup-exception field that only `Start` reads, leaving a live worker whose
+commands queued forever. Startup failures still surface from `Start` itself.
+
 ### Inner Completion Waits
 
 Two commands hold the STA while waiting for a COM event they just provoked: the
 unary write path waits for its `OnWriteComplete`
-(`MxAccessWriteCompletionCache.TryWaitForCompletion`), and `ReadBulk` waits per
+(`MxAccessWriteCompletionCache.TryWaitForCompletion`), and `ReadBulk` waits for
-tag for the first `OnDataChange` (`MxAccessValueCache.TryWaitForUpdate`). Both
+the first `OnDataChange` of every snapshot tag
+(`MxAccessValueCache.TryWaitForUpdates`). Both
 run the same loop shape as the outer pump, for the same reason — the event they
 are waiting for *is* a Windows message, so the thread must keep dispatching to
 receive it:
@@ -297,7 +321,7 @@ The idle slice is a Win32 wait (`StaWaitHelper.WaitForSignalOrMessages`), never
 `Thread.Sleep`. A sleeping STA pumps no messages, so a sleep-polled loop could
 only dispatch the awaited COM event at poll-tick granularity while stalling
 *every other* event for the same tick — up to 1.5 s for a write completion and
-up to `timeout_ms` per tag for `ReadBulk`. The Win32 wait returns the instant a
+up to `timeout_ms` for `ReadBulk`. The Win32 wait returns the instant a
 message needs pumping, so the apartment dispatches continuously for the whole
 wait. Each cache also sets an `AutoResetEvent` from its update path (outside the
 cache lock) so a cross-thread producer wakes the waiter immediately; in the live
@@ -317,8 +341,29 @@ nothing wakes the wait — a process with no STA message queue (unit tests drive
 these caches from ordinary threads, standing in for the STA by updating the
 cache from a fake `pumpStep`) must not block for a full poll interval. Timeouts,
 deadline math, and return values are unchanged by the wait mechanism: an expired
-write wait still yields the empty-`statuses` unconfirmed reply, and an expired
+write wait still yields the empty-`statuses` unconfirmed reply, and every
-per-tag `ReadBulk` wait still reports its own timeout.
+`ReadBulk` tag that did not deliver before the deadline still reports its own
+timeout.
+
+`ReadBulk` runs its snapshot lifecycle in phases so the STA is held for at most
+one `timeout_ms` per command, not one per tag: (1) every tag is validated and
+probed against the cached-and-advised fast path; (2) every remaining tag is
+added and advised; (3) one wait with a shared deadline collects the first
+`OnDataChange` per snapshot handle and returns as soon as all have delivered;
+(4) every snapshot handle is unadvised and removed, best-effort. Per-tag
+results are unchanged: request order, the blank-address failure, the cached
+result, an `AddItem` failure (item handle 0) or `Advise` failure (its own item
+handle) failing only that tag, and the `ReadBulk timed out after <n> ms waiting
+for first OnDataChange.` message for each tag that never delivered. The cache
+probes all run before any snapshot `AddItem`, so a later tag in the same
+request can never be served from an earlier tag's snapshot subscription.
+When MXAccess returns the same item handle for two tags in one batch (a
+duplicate address), the handle is advised, unadvised, and removed once, and
+every duplicate receives the shared outcome. Phases 2 and 4 make one or two
+COM calls per tag without waiting, so they call the pump step every
+`ReadBulkPumpInterval` (16) tags; a large batch therefore keeps dispatching
+inbound callbacks and refreshing `LastStaActivityUtc` instead of drifting
+toward `HeartbeatStuckCeiling`.
 
 The write wait's budget is `MxGateway:Worker:WriteCompletionWaitMilliseconds`
 (default 1500). It applies to `Write`, `Write2`, `WriteSecured`, and
@@ -351,6 +396,23 @@ native client correlating by handle has the identical ambiguity). The exposure
 is bounded: `statuses` is an enrichment field, and `Hresult`/`ProtocolStatus`
 stay acceptance-only regardless of which completion (if any) was attributed.
 
+**The alarm ack writer narrows the window for acknowledgements.**
+`AlarmAckWriter` registers its own MXAccess server with its own
+`MxAccessWriteCompletionCache` and reuses one advised `AckMsg` item per alarm
+(least-recently-used, capped at 256). After an ack ends unconfirmed it marks the
+item dirty, and the next ack to that alarm removes and re-adds the item so the
+write waits on a fresh item handle the stale completion cannot match. This
+relies on MXAccess never reusing an item handle number within one
+registration, which a live probe confirmed (see "Item handle reuse" in
+[Alarm Probe Findings](./AlarmProbeFindings.md)). Every
+drop of an item (dirty re-add, eviction, or a failed `AddItem`/`Advise`/`Write`)
+also removes that handle's completion entry, and tearing down the registration
+removes every entry for its server handle, so the cache does not grow by one
+entry per dropped handle for the life of the worker. The writer records a
+completion only for an item handle it currently holds on its live
+registration, so a late `OnWriteComplete` for a dropped handle cannot recreate
+its entry.
+
 ## Live alarm watch-list replacement
 
 An auto-mode `SubscribeAlarms` with `fallback_enabled = true` always builds a
@@ -654,7 +716,20 @@ The worker records plain and supervisory advice separately only after the COM
 call returns normally. Successful `UnAdvise` removes all tracked advice for the
 server and item pair because the public MXAccess cleanup method has no plain
 versus supervisory selector. Successful `RemoveItem` and `Unregister` also clear
-related advice state from the worker registry. Failed advice and cleanup calls
+related advice state from the worker registry, and evict the per-handle
+caches the event sink fills: `RemoveItem` drops the handle pair's
+`MxAccessValueCache` and `MxAccessWriteCompletionCache` entries, and
+`Unregister` drops every entry recorded under the server handle (MXAccess
+removes the server's items with it, so the per-item eviction never runs for
+them). Without this, an `AddItem` → `Write` → `RemoveItem` churn grew the
+completion cache by one entry per cycle for the life of the worker.
+`AddItem`, `AddItem2`, and `AddBufferedItem` also drop both cache entries for
+the handle they receive when the registry does not already track it. An
+`OnDataChange` or `OnWriteComplete` that was queued when `RemoveItem` or
+`Unregister` ran is dispatched after the eviction and recreates the entry;
+when MXAccess later reuses that item id, `ReadBulk`'s cached path would
+otherwise serve the stale value as `was_cached = true`. A handle the
+registry still tracks belongs to a live item, so its entries are kept. Failed advice and cleanup calls
 leave registry state unchanged so diagnostics continue to reflect the last
 successful MXAccess-owned state transition.
 
@@ -888,6 +963,23 @@ the STA more than 75 s without pumping *should* fault — that is the ceiling's
 contract). The flag is named generically so any future non-dispatcher STA work
 reuses it.
 
+A `PollOnce` that throws does not fault the session on the first failure. The
+poll loop counts consecutive failures and records the alarm-poll fault
+(`MxaccessEventConversionFailed`) only when the count reaches
+`MxAccessStaSession.DefaultAlarmPollFailureThreshold` (3, the same default as
+`AlarmFailoverConfig.consecutive_failure_threshold`); a successful poll resets
+the count. Each failure below the threshold writes one
+`level=Warning event=AlarmPollFailed` line to stderr carrying the exception
+type, HRESULT, and counts only — the exception message is left out because it
+can echo alarm or tag names. A single transient COM failure in
+`GetXmlCurrentAlarms2` used to fault the whole session, taking every data
+subscription with it; a persistent failure, including an STA-affinity
+regression that fails every tick, still faults within three poll intervals.
+A poll failure seen after the STA loop itself has died (`StaRuntime.LoopFault`
+is set) ends the poll loop without recording anything: the loop-fault handler
+has already recorded the `StaHung` fault, and the failure is only the
+fast-fail `InvokeAsync` returns afterwards.
+
 The in-flight-command suppression itself is bounded by
 `WorkerPipeSessionOptions.HeartbeatStuckCeiling` (default 75 seconds = 5 ×
 `HeartbeatGrace`). The motivating case for the suppression is a legitimately

docs/WorkerBootstrap.md

@@ -117,8 +117,13 @@ public static WorkerBootstrapResult Failure(WorkerExitCode exitCode, IEnumerable
 | `MissingNonce` | 4 | The `MXGATEWAY_WORKER_NONCE` environment variable is null, empty, or whitespace. |
 | `PipeConnectionFailed` | 5 | An `IOException` or `TimeoutException` escapes the pipe client. |
 | `ProtocolViolation` | 6 | A `WorkerFrameProtocolException` escapes the pipe client. |
+| `UnhandledException` | 7 | An exception reaches `AppDomain.CurrentDomain.UnhandledException` — thrown on a thread outside `WorkerApplication.Run`'s `try/catch` (a background task continuation, a timer, a COM callback thread). |
 
-`InvalidArguments`, `InvalidProtocolVersion`, and `MissingNonce` originate in the parser; the others originate in `WorkerApplication.Run`'s `try/catch` around the pipe client.
+`InvalidArguments`, `InvalidProtocolVersion`, and `MissingNonce` originate in the parser; `UnhandledException` originates in the process-wide hook described below; the others originate in `WorkerApplication.Run`'s `try/catch` around the pipe client.
+
+### Unhandled-exception hook
+
+The production entry point, `WorkerApplication.Run(string[])`, subscribes `AppDomain.CurrentDomain.UnhandledException` before it parses arguments. The handler (`WorkerApplication.HandleUnhandledException`) logs one `WorkerUnhandledException` error line and calls `Environment.Exit(7)`. Without it, an exception on a thread that `Run` does not own ended the process with the CLR's crash code and no worker log line, so the gateway saw an unexplained exit. The line carries `exit_code`, `exception_type`, and `is_terminating` only: the exception message and stack are left out because they can carry tag values or credentials passed through a command, the same rule every other worker failure line follows. A failure inside the logger is swallowed so the exit always happens. The injectable `Run` overloads that tests call do not install the hook, because it is process-global.
 
 ## Environment Abstraction
 

docs/WorkerSta.md

@@ -86,10 +86,16 @@ StaWorkItem<T> workItem = new(command, cancellationToken);
 
 lock (gate)
 {
+    if (loopFault is not null)
+    {
+        return Task.FromException<T>(new InvalidOperationException(
+            "The worker STA loop terminated unexpectedly.",
+            loopFault));
+    }
+
     if (shutdownRequested)
     {
-        return Task.FromException<T>(
+        return Task.FromException<T>(new StaRuntimeShutdownException());
-            new InvalidOperationException("The worker STA runtime is shutting down."));
     }
 
     commandQueue.Enqueue(workItem);
@@ -101,7 +107,11 @@ return workItem.Task;
 
 `StaWorkItem<T>` uses an `Interlocked.CompareExchange` on `started` so that exactly one of three outcomes happens: the STA thread executes the delegate, an external `CancellationToken` cancellation fires first, or `CancelBeforeExecution` runs during shutdown. `Execute` runs on the STA thread, sets `Completion` from `command()`, and propagates exceptions through `TrySetException` so the awaiting caller observes them.
 
-`ProcessQueuedCommands` is the only consumer of the queue and runs on the STA thread, so each work item is guaranteed to execute on the apartment that owns the COM object.
+`ProcessQueuedCommands` is the only consumer of the queue and runs on the STA thread, so each work item is guaranteed to execute on the apartment that owns the COM object. It calls `PumpPendingMessages` between work items whenever more work is queued, so a backlog of commands cannot hold inbound COM callbacks until the whole queue drains.
+
+### Loop failure after start
+
+An exception thrown by the loop after `Start()` returned — `WaitForWorkOrMessages` reporting `WAIT_FAILED`, or an exception escaping a dispatched message — is not a startup failure, and nothing reads `startupException` after `Start()`. `ThreadMain` instead stores it as `LoopFault` under `gate`, so every later `InvokeAsync` fails immediately with an `InvalidOperationException` whose inner exception is the loop fault, and work already queued is cancelled by the `finally` block. It then invokes the handler registered with `RegisterLoopFaultHandler`; a handler registered after the loop died runs immediately on the registering thread. Neither happens when `Shutdown` was already requested, because a loop that throws during a deliberate teardown is not a worker fault. `MxAccessStaSession` registers a handler that records a `StaHung` `WorkerFault` on the event queue, which the IPC drain loop sends to the gateway before ending the pipe session and the worker process.
 
 ## Command Dispatch
 

gateway.md

@@ -140,8 +140,9 @@ one gateway-owned worker session subscribes the configured AVEVA alarm
 provider, caches the active-alarm set (reconciled periodically against the
 worker's snapshot), and fans it out to every client through the session-less
 `StreamAlarms` RPC — the stream opens with the current active-alarm snapshot,
-then streams live transitions. `AcknowledgeAlarm` is session-less and routes
+then streams live transitions. `AcknowledgeAlarm` is session-less, routes
-through the monitor. Clients never open a worker session to see alarms, and
+through the monitor, and carries the operator's credentials (see
+"Acknowledge" below). Clients never open a worker session to see alarms, and
 alarm monitoring is independent of client lifecycle; the monitor re-opens its
 session if the worker faults. Gated by `MxGateway:Alarms:Enabled` — see
 `docs/DesignDecisions.md` for why this reverses the v1 single-subscriber rule
@@ -221,20 +222,22 @@ forwards only events the worker emits and never synthesizes transitions
 itself. The synthesis rules are documented in
 `docs/AlarmClientDiscovery.md`.
 
-**Acknowledge in subtag mode:** the ack-by-name path writes the operator
+**Acknowledge:** the gateway acknowledges an alarm by writing the operator's
-comment to the alarm attribute's ack-comment subtag. The write performs the
+comment to the alarm attribute's `AckMsg` (subtag mode: the configured
-ack. If the attribute has no writable ack-comment subtag configured, the RPC
+ack-comment subtag) over MXAccess, after `AuthenticateUser` as the requesting
-returns `FailedPrecondition`. In alarm-manager mode, `AlarmAckByName` is
+operator (`operator_user`, `operator_password`, optional `operator_domain`).
-used as before.
+The AVEVA runtime treats that write as the acknowledgement and historizes it
-
+with the authenticated account's name, domain and full name; the Historian's
-**Acceptance is not application:** the ack is forwarded to the provider, and a
+OperatorNode is the ack writer's MXAccess client name
-successful return means the provider accepted the call, not that the
+(`MxGateway:Alarms:AckClientName`). The worker waits a bounded time for
-acknowledgement was applied. On galaxies whose alarmed attributes carry a
+`OnWriteComplete`: an OK status is reported as success, a non-OK status as
-non-free-access security classification, `AlarmAckByName` returns `rc=0` and the
+`MxaccessFailure` with the status detail, and no completion as success with a
-alarm stays `UNACK_ALM` — it carries an operator name, not an authenticated
+diagnostic noting the write was sent but unconfirmed. An unknown alarm or one
-identity. (Observed on the probe rig; the mechanism is inferred.) Confirm an ack
+with no ack target (non-Galaxy provider, no ack-comment subtag) is reported as
-by the resulting transition, never by the return code. See
+`InvalidRequest` with `hresult` -1. Alarm-client ack calls
-`docs/AlarmProbeFindings.md`.
+(`AlarmAckByName`/`AlarmAckByGUID`, the managed `AlarmClient`, WNAL) are not
+used: live tests showed a headless consumer's acks are accepted and silently
+dropped (see `docs/AlarmProbeFindings.md`).
 
 **Degraded state visibility:** every subtag-mode transition carries
 `degraded = true` and `source_provider = ALARM_PROVIDER_MODE_SUBTAG` on the
@@ -566,8 +569,15 @@ returns one result per requested entry — per-entry failures populate
   advised (`was_cached = true`, no subscription side-effects), or by
   taking a full `AddItem` + `Advise` + wait-for-first-OnDataChange +
   `UnAdvise` + `RemoveItem` snapshot lifecycle when no live subscription
-  exists (`was_cached = false`). Per-tag timeouts surface as
+  exists (`was_cached = false`). `timeout_ms` is one shared deadline for
-  `was_successful = false` rather than throwing. The cache lives on the
+  the whole command, not a per-tag budget: the worker adds and advises
+  every uncached tag first, waits once for all of them, then tears them
+  all down, so the STA is held for at most one `timeout_ms` (plus the
+  add/advise and teardown COM calls, during which it pumps every 16 tags).
+  Each tag that has not delivered by the deadline surfaces its own
+  `was_successful = false` timeout rather than throwing. Duplicate
+  addresses that MXAccess resolves to the same item handle share one
+  subscription and receive the same result. The cache lives on the
   worker's `MxAccessValueCache`, populated by `MxAccessBaseEventSink` on
   every `OnDataChange` after the event clears the outbound queue.
   Required scope: `invoke:read`. `timeout_ms == 0` uses the worker's
@@ -1028,8 +1038,9 @@ Internal worker IPC:
 - gateway validates worker executable path and version,
 - no secrets in command line when avoidable.
 
-Credential-sensitive commands such as `AuthenticateUser` and `WriteSecured`
+Credential-sensitive commands such as `AuthenticateUser`, `WriteSecured` and
-must not log passwords or raw credential values.
+`AcknowledgeAlarm` (its `operator_password`) must not log passwords or raw
+credential values.
 
 ## Observability
 

src/ZB.MOM.WW.MxGateway.IntegrationTests/WorkerLiveMxAccessSmokeTests.cs

@@ -1052,6 +1052,231 @@ public sealed class WorkerLiveMxAccessSmokeTests(ITestOutputHelper output)
         }
     }
 
+    /// <summary>
+    /// TST-05: the real worker answers the four non-terminating control commands off-STA in
+    /// <c>WorkerPipeSession.HandleControlCommandAsync</c>. The default suite only ever sees the
+    /// FakeWorkerHarness canned replies, so this drives <c>Ping</c>, <c>GetSessionState</c>,
+    /// <c>GetWorkerInfo</c>, and <c>DrainEvents</c> through the gateway against a live worker and
+    /// asserts each reply carries real worker state rather than an <c>INVALID_REQUEST</c>.
+    /// </summary>
+    /// <returns>A task that represents the asynchronous operation.</returns>
+    [LiveMxAccessFact]
+    public async Task GatewaySession_WithLiveWorker_ControlCommands_ReturnRealWorkerState()
+    {
+        string workerExecutablePath = IntegrationTestEnvironment.ResolveLiveMxAccessWorkerExecutablePath();
+        Assert.True(
+            File.Exists(workerExecutablePath),
+            $"Live MXAccess worker executable was not found at {workerExecutablePath}. Build the worker or set {IntegrationTestEnvironment.LiveMxAccessWorkerExecutableVariableName}.");
+
+        TestWorkerProcessFactory processFactory = new(output);
+        await using GatewayServiceFixture fixture = new(workerExecutablePath, processFactory, output);
+
+        string? sessionId = null;
+
+        try
+        {
+            OpenSessionReply openReply = await fixture.Service.OpenSession(
+                new OpenSessionRequest
+                {
+                    ClientSessionName = "live-mxaccess-control",
+                    ClientCorrelationId = "live-open-control",
+                    CommandTimeout = Duration.FromTimeSpan(CommandTimeout),
+                },
+                new TestServerCallContext()).ConfigureAwait(false);
+
+            sessionId = openReply.SessionId;
+            Assert.Equal(ProtocolStatusCode.Ok, openReply.ProtocolStatus.Code);
+            Assert.True(openReply.WorkerProcessId > 0);
+
+            MxCommandReply pingReply = await fixture.Service.Invoke(
+                CreateControlRequest(sessionId, "live-ping", new MxCommand
+                {
+                    Kind = MxCommandKind.Ping,
+                    Ping = new PingCommand { Message = "live-ping-echo" },
+                }),
+                new TestServerCallContext()).ConfigureAwait(false);
+            LogReply("Ping", pingReply);
+            Assert.Equal(ProtocolStatusCode.Ok, pingReply.ProtocolStatus.Code);
+            Assert.Equal(MxCommandKind.Ping, pingReply.Kind);
+
+            MxCommandReply stateReply = await fixture.Service.Invoke(
+                CreateControlRequest(sessionId, "live-get-session-state", new MxCommand
+                {
+                    Kind = MxCommandKind.GetSessionState,
+                    GetSessionState = new GetSessionStateCommand(),
+                }),
+                new TestServerCallContext()).ConfigureAwait(false);
+            LogReply("GetSessionState", stateReply);
+            Assert.Equal(ProtocolStatusCode.Ok, stateReply.ProtocolStatus.Code);
+            Assert.Equal(MxCommandKind.GetSessionState, stateReply.Kind);
+            Assert.NotNull(stateReply.SessionState);
+            Assert.Equal(SessionState.Ready, stateReply.SessionState.State);
+
+            MxCommandReply infoReply = await fixture.Service.Invoke(
+                CreateControlRequest(sessionId, "live-get-worker-info", new MxCommand
+                {
+                    Kind = MxCommandKind.GetWorkerInfo,
+                    GetWorkerInfo = new GetWorkerInfoCommand(),
+                }),
+                new TestServerCallContext()).ConfigureAwait(false);
+            LogReply("GetWorkerInfo", infoReply);
+            Assert.Equal(ProtocolStatusCode.Ok, infoReply.ProtocolStatus.Code);
+            Assert.Equal(MxCommandKind.GetWorkerInfo, infoReply.Kind);
+            Assert.NotNull(infoReply.WorkerInfo);
+            output.WriteLine(
+                $"GetWorkerInfo pid={infoReply.WorkerInfo.WorkerProcessId} version={infoReply.WorkerInfo.WorkerVersion} "
+                + $"progid={infoReply.WorkerInfo.MxaccessProgid} clsid={infoReply.WorkerInfo.MxaccessClsid}");
+            Assert.Equal(openReply.WorkerProcessId, infoReply.WorkerInfo.WorkerProcessId);
+            Assert.False(string.IsNullOrWhiteSpace(infoReply.WorkerInfo.WorkerVersion));
+            Assert.False(string.IsNullOrWhiteSpace(infoReply.WorkerInfo.MxaccessProgid));
+            Assert.False(string.IsNullOrWhiteSpace(infoReply.WorkerInfo.MxaccessClsid));
+
+            // No StreamEvents subscriber is attached, so whatever the worker queued since start
+            // is still drainable; the reply must carry the DrainEvents payload (possibly empty).
+            MxCommandReply drainReply = await fixture.Service.Invoke(
+                CreateControlRequest(sessionId, "live-drain-events", new MxCommand
+                {
+                    Kind = MxCommandKind.DrainEvents,
+                    DrainEvents = new DrainEventsCommand { MaxEvents = 16 },
+                }),
+                new TestServerCallContext()).ConfigureAwait(false);
+            LogReply("DrainEvents", drainReply);
+            Assert.Equal(ProtocolStatusCode.Ok, drainReply.ProtocolStatus.Code);
+            Assert.Equal(MxCommandKind.DrainEvents, drainReply.Kind);
+            Assert.Equal(MxCommandReply.PayloadOneofCase.DrainEvents, drainReply.PayloadCase);
+            Assert.True(drainReply.DrainEvents.Events.Count <= 16);
+            output.WriteLine($"DrainEvents drained={drainReply.DrainEvents.Events.Count}");
+
+            // A control command must not disturb the session: it is still Ready afterwards.
+            Assert.True(fixture.TryGetSession(sessionId, out GatewaySession? session));
+            Assert.Equal(SessionState.Ready, session!.State);
+        }
+        finally
+        {
+            await ShutDownAsync(fixture, processFactory, sessionId, streamTask: null).ConfigureAwait(false);
+        }
+    }
+
+    /// <summary>
+    /// TST-05: <c>ShutdownWorker</c> writes its OK reply before the worker shuts down, so the
+    /// reply reaches the caller (a reply written after the pipe closed would surface as a
+    /// pipe-disconnected failure instead), the worker process then exits on its own, and a later
+    /// command on the session does not succeed. Uses its own fixture and worker because it
+    /// terminates the worker.
+    /// </summary>
+    /// <returns>A task that represents the asynchronous operation.</returns>
+    [LiveMxAccessFact]
+    public async Task GatewaySession_WithLiveWorker_ShutdownWorker_RepliesOkThenWorkerExits()
+    {
+        string workerExecutablePath = IntegrationTestEnvironment.ResolveLiveMxAccessWorkerExecutablePath();
+        Assert.True(
+            File.Exists(workerExecutablePath),
+            $"Live MXAccess worker executable was not found at {workerExecutablePath}. Build the worker or set {IntegrationTestEnvironment.LiveMxAccessWorkerExecutableVariableName}.");
+
+        TestWorkerProcessFactory processFactory = new(output);
+        await using GatewayServiceFixture fixture = new(workerExecutablePath, processFactory, output);
+        using RecordingServerStreamWriter<MxEvent> eventWriter = new();
+
+        string? sessionId = null;
+        Task? streamTask = null;
+        using CancellationTokenSource streamCancellation = new();
+
+        try
+        {
+            OpenSessionReply openReply = await fixture.Service.OpenSession(
+                new OpenSessionRequest
+                {
+                    ClientSessionName = "live-mxaccess-shutdown-worker",
+                    ClientCorrelationId = "live-open-shutdown-worker",
+                    CommandTimeout = Duration.FromTimeSpan(CommandTimeout),
+                },
+                new TestServerCallContext()).ConfigureAwait(false);
+
+            sessionId = openReply.SessionId;
+            Assert.Equal(ProtocolStatusCode.Ok, openReply.ProtocolStatus.Code);
+
+            // The gateway observes a worker's pipe closing through the event-stream read loop
+            // (the same path the abnormal-exit fact relies on); without a subscriber the session
+            // would sit in Ready until the heartbeat grace expired.
+            streamTask = fixture.Service.StreamEvents(
+                new StreamEventsRequest { SessionId = sessionId },
+                eventWriter,
+                new TestServerCallContext(streamCancellation.Token));
+
+            MxCommandReply shutdownReply = await fixture.Service.Invoke(
+                CreateControlRequest(sessionId, "live-shutdown-worker", new MxCommand
+                {
+                    Kind = MxCommandKind.ShutdownWorker,
+                    ShutdownWorker = new ShutdownWorkerCommand
+                    {
+                        GracePeriod = Duration.FromTimeSpan(TimeSpan.FromSeconds(5)),
+                    },
+                }),
+                new TestServerCallContext()).ConfigureAwait(false);
+            LogReply("ShutdownWorker", shutdownReply);
+            Assert.Equal(ProtocolStatusCode.Ok, shutdownReply.ProtocolStatus.Code);
+            Assert.Equal(MxCommandKind.ShutdownWorker, shutdownReply.Kind);
+
+            // The worker exits on its own — no CloseSession, no kill. WaitForProcessesAsync
+            // throws on timeout, which fails the test.
+            await processFactory.WaitForProcessesAsync(StreamShutdownTimeout).ConfigureAwait(false);
+
+            // The session can no longer serve commands once its worker is gone: either the
+            // gateway rejects the call outright or the reply is not Ok.
+            MxCommandReply? afterReply = null;
+            Exception? afterException = null;
+            try
+            {
+                afterReply = await fixture.Service.Invoke(
+                    CreateControlRequest(sessionId, "live-ping-after-shutdown", new MxCommand
+                    {
+                        Kind = MxCommandKind.Ping,
+                        Ping = new PingCommand(),
+                    }),
+                    new TestServerCallContext()).ConfigureAwait(false);
+                LogReply("PingAfterShutdown", afterReply);
+            }
+            catch (Exception exception)
+            {
+                afterException = exception;
+                output.WriteLine($"PingAfterShutdown rejected: {exception.GetType().Name}: {exception.Message}");
+            }
+
+            Assert.True(
+                afterException is not null || afterReply!.ProtocolStatus.Code != ProtocolStatusCode.Ok,
+                "A command sent after ShutdownWorker must not succeed against the exited worker.");
+
+            // Logged, not asserted: the gateway's WorkerClient moves to Closed on the shutdown
+            // ack, but GatewaySession.State is not driven from that transition today, so the
+            // public session state still reads Ready here (observed live on windev). That is a
+            // gateway-side gap, not a worker one — this fact pins the worker half of the
+            // contract: OK reply first, then a clean exit.
+            string observedState = fixture.TryGetSession(sessionId, out GatewaySession? session)
+                ? session.State.ToString()
+                : "<removed>";
+            output.WriteLine($"ShutdownWorker observed_session_state={observedState}");
+        }
+        finally
+        {
+            // The worker is gone; a CloseSession round-trip would only log a cleanup failure.
+            streamCancellation.Cancel();
+            await ShutDownAsync(fixture, processFactory, sessionId: null, streamTask).ConfigureAwait(false);
+        }
+    }
+
+    private static MxCommandRequest CreateControlRequest(
+        string sessionId,
+        string correlationId,
+        MxCommand command)
+    {
+        return new MxCommandRequest
+        {
+            SessionId = sessionId,
+            ClientCorrelationId = correlationId,
+            Command = command,
+        };
+    }
+
     /// <summary>
     /// Closes the session and drains the event stream / worker processes without letting a
     /// cleanup timeout mask the original failure from the test body.

src/ZB.MOM.WW.MxGateway.Server/Alarms/GatewayAlarmMonitor.cs

@@ -1169,7 +1169,7 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
         return string.Empty;
     }
 
-    private static MxCommand? BuildAcknowledgeCommand(AcknowledgeAlarmRequest request, out string? parseError)
+    internal static MxCommand? BuildAcknowledgeCommand(AcknowledgeAlarmRequest request, out string? parseError)
     {
         parseError = null;
         if (string.IsNullOrWhiteSpace(request.AlarmFullReference))
@@ -1180,6 +1180,8 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
 
         string comment = request.Comment ?? string.Empty;
         string operatorUser = request.OperatorUser ?? string.Empty;
+        string operatorPassword = request.OperatorPassword ?? string.Empty;
+        string operatorDomain = request.OperatorDomain ?? string.Empty;
 
         if (Guid.TryParse(request.AlarmFullReference, out Guid guid))
         {
@@ -1192,7 +1194,8 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
                     Comment = comment,
                     OperatorUser = operatorUser,
                     OperatorNode = string.Empty,
-                    OperatorDomain = string.Empty,
+                    OperatorDomain = operatorDomain,
+                    OperatorPassword = operatorPassword,
                     OperatorFullName = string.Empty,
                 },
             };
@@ -1211,7 +1214,8 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
                     Comment = comment,
                     OperatorUser = operatorUser,
                     OperatorNode = string.Empty,
-                    OperatorDomain = string.Empty,
+                    OperatorDomain = operatorDomain,
+                    OperatorPassword = operatorPassword,
                     OperatorFullName = string.Empty,
                 },
             };

src/ZB.MOM.WW.MxGateway.Server/Configuration/AlarmsOptions.cs

@@ -73,6 +73,13 @@ public sealed class AlarmsOptions
     /// </summary>
     public int MaxAlarmsPerFetch { get; init; } = 1024;
 
+    /// <summary>
+    ///     MXAccess client name the alarm worker's ack writer registers with. The
+    ///     AVEVA Historian records it as each acknowledgement's OperatorNode.
+    ///     Conveyed to workers via MXGATEWAY_ALARM_ACK_CLIENT_NAME.
+    /// </summary>
+    public string AckClientName { get; init; } = "MxGateway";
+
     /// <summary>
     ///     Configuration for the alarm-manager ↔ subtag fallback mechanism:
     ///     operating mode, failure-detection thresholds, discovery, and subtag

src/ZB.MOM.WW.MxGateway.Server/Configuration/GatewayOptionsValidator.cs

@@ -488,6 +488,13 @@ public sealed class GatewayOptionsValidator : OptionsValidatorBase<GatewayOption
                 $"MxGateway:Alarms:MaxAlarmsPerFetch must be between {MinimumMaxAlarmsPerFetch} and {MaximumMaxAlarmsPerFetch}.");
         }
 
+        if (string.IsNullOrWhiteSpace(options.AckClientName)
+            || options.AckClientName.Length > 64
+            || options.AckClientName.Any(char.IsControl))
+        {
+            builder.Add("MxGateway:Alarms:AckClientName must be 1-64 printable characters.");
+        }
+
         if (!options.Enabled)
         {
             return;

src/ZB.MOM.WW.MxGateway.Server/Diagnostics/GatewayLogRedactor.cs

@@ -12,7 +12,10 @@ public static class GatewayLogRedactor
     {
         "AuthenticateUser",
         "WriteSecured",
-        "WriteSecured2"
+        "WriteSecured2",
+        // Carry operator_password.
+        "AcknowledgeAlarm",
+        "AcknowledgeAlarmByName"
     };
 
     /// <summary>

src/ZB.MOM.WW.MxGateway.Server/Diagnostics/GatewayLogRedactorSeam.cs

@@ -33,6 +33,14 @@ public sealed class GatewayLogRedactorSeam : ILogRedactor
         }
 
         RedactCommandValue(properties);
+
+        foreach (string key in properties.Keys.ToList())
+        {
+            if (key.Contains("password", StringComparison.OrdinalIgnoreCase) && properties[key] is not null)
+            {
+                properties[key] = GatewayLogRedactor.RedactedValue;
+            }
+        }
     }
 
     /// <summary>

src/ZB.MOM.WW.MxGateway.Server/Grpc/MxAccessGatewayService.cs

@@ -197,6 +197,11 @@ public sealed class MxAccessGatewayService(
                 throw new RpcException(new Status(StatusCode.InvalidArgument, "alarm_full_reference is required."));
             }
 
+            if (string.IsNullOrWhiteSpace(request.OperatorUser))
+            {
+                throw new RpcException(new Status(StatusCode.InvalidArgument, "operator_user is required."));
+            }
+
             return await alarmService.AcknowledgeAsync(request, context.CancellationToken)
                 .ConfigureAwait(false);
         }

src/ZB.MOM.WW.MxGateway.Server/Workers/WorkerProcessLauncher.cs

@@ -223,6 +223,8 @@ public sealed class WorkerProcessLauncher : IWorkerProcessLauncher
         startInfo.Environment[WorkerLaunchEnvironment.MaxAlarmsPerFetch] =
             _alarmsOptions.MaxAlarmsPerFetch.ToString(System.Globalization.CultureInfo.InvariantCulture);
 
+        startInfo.Environment[WorkerLaunchEnvironment.AlarmAckClientName] = _alarmsOptions.AckClientName;
+
         commandLine = new WorkerProcessCommandLine(executablePath, arguments);
 
         return startInfo;

src/ZB.MOM.WW.MxGateway.Server/ZB.MOM.WW.MxGateway.Server.csproj

@@ -32,6 +32,9 @@
          native 2.1.11; 2.1.12 patches it. Bumping Sqlite does not clear it. -->
     <PackageReference Include="SQLitePCLRaw.lib.e_sqlite3" Version="2.1.12" />
     <PackageReference Include="Microsoft.Data.SqlClient" Version="6.0.2" />
+    <!-- Security pin: GHSA-846f-f764-76mx / GHSA-2fmv-rv2c-59x7. Microsoft.Data.SqlClient 6.0.2
+         pulls the vulnerable System.Security.Cryptography.Pkcs 9.0.4; 10.0.13 patches it. -->
+    <PackageReference Include="System.Security.Cryptography.Pkcs" Version="10.0.13" />
     <PackageReference Include="Polly.Core" Version="8.6.6" />
   </ItemGroup>
 

src/ZB.MOM.WW.MxGateway.Server/appsettings.json

@@ -88,7 +88,8 @@
       "DefaultArea": "",
       "ReconcileIntervalSeconds": 30,
       "PollIntervalMilliseconds": 500,
-      "MaxAlarmsPerFetch": 1024
+      "MaxAlarmsPerFetch": 1024,
+      "AckClientName": "MxGateway"
     }
   }
 }

src/ZB.MOM.WW.MxGateway.Tests/Alarms/GatewayAlarmMonitorAcknowledgeTests.cs

@@ -0,0 +1,46 @@
+using ZB.MOM.WW.MxGateway.Contracts.Proto;
+using ZB.MOM.WW.MxGateway.Server.Alarms;
+
+namespace ZB.MOM.WW.MxGateway.Tests.Alarms;
+
+public sealed class GatewayAlarmMonitorAcknowledgeTests
+{
+    /// <summary>A GUID-referenced ack carries the operator identity and password to the worker command.</summary>
+    [Fact]
+    public void BuildAcknowledgeCommand_GuidReference_ForwardsOperatorCredentials()
+    {
+        MxCommand? command = GatewayAlarmMonitor.BuildAcknowledgeCommand(new AcknowledgeAlarmRequest
+        {
+            AlarmFullReference = "BCC47053-9542-4D65-BDAA-BCDEA6A32A73",
+            Comment = "checked",
+            OperatorUser = "alice",
+            OperatorDomain = "CORP",
+            OperatorPassword = "pw-secret",
+        }, out string? error);
+
+        Assert.Null(error);
+        Assert.Equal(MxCommandKind.AcknowledgeAlarm, command!.Kind);
+        Assert.Equal("alice", command.AcknowledgeAlarmCommand.OperatorUser);
+        Assert.Equal("CORP", command.AcknowledgeAlarmCommand.OperatorDomain);
+        Assert.Equal("pw-secret", command.AcknowledgeAlarmCommand.OperatorPassword);
+    }
+
+    /// <summary>A name-referenced ack carries the operator identity and password to the by-name command.</summary>
+    [Fact]
+    public void BuildAcknowledgeCommand_NameReference_ForwardsOperatorCredentials()
+    {
+        MxCommand? command = GatewayAlarmMonitor.BuildAcknowledgeCommand(new AcknowledgeAlarmRequest
+        {
+            AlarmFullReference = "Galaxy!TestArea.TestMachine_001.TestAlarm001",
+            OperatorUser = "alice",
+            OperatorDomain = "CORP",
+            OperatorPassword = "pw-secret",
+        }, out _);
+
+        Assert.Equal(MxCommandKind.AcknowledgeAlarmByName, command!.Kind);
+        Assert.Equal("TestMachine_001.TestAlarm001", command.AcknowledgeAlarmByNameCommand.AlarmName);
+        Assert.Equal("alice", command.AcknowledgeAlarmByNameCommand.OperatorUser);
+        Assert.Equal("CORP", command.AcknowledgeAlarmByNameCommand.OperatorDomain);
+        Assert.Equal("pw-secret", command.AcknowledgeAlarmByNameCommand.OperatorPassword);
+    }
+}

src/ZB.MOM.WW.MxGateway.Tests/Configuration/GatewayOptionsValidatorTests.cs

@@ -164,6 +164,45 @@ public sealed class GatewayOptionsValidatorTests
             Tls = source.Tls,
         };
 
+    /// <summary>Verifies AckClientName defaults to "MxGateway" and passes validation.</summary>
+    [Fact]
+    public void Validate_Succeeds_WithDefaultAckClientName()
+    {
+        Assert.Equal("MxGateway", new AlarmsOptions().AckClientName);
+
+        ValidateOptionsResult result = new GatewayOptionsValidator()
+            .Validate(null, CloneWithAlarms(ValidOptions(), new AlarmsOptions()));
+        Assert.True(result.Succeeded);
+    }
+
+    /// <summary>Verifies an empty, over-long, or control-character AckClientName fails validation.</summary>
+    /// <param name="name">Client name under test.</param>
+    [Theory]
+    [InlineData("")]
+    [InlineData("   ")]
+    [InlineData("bad\nname")]
+    public void Validate_Fails_WhenAckClientNameInvalid(string name)
+    {
+        ValidateOptionsResult result = new GatewayOptionsValidator()
+            .Validate(null, CloneWithAlarms(ValidOptions(), new AlarmsOptions { AckClientName = name }));
+
+        Assert.False(result.Succeeded);
+        Assert.Contains(
+            result.Failures!,
+            f => f.Contains("MxGateway:Alarms:AckClientName", StringComparison.Ordinal));
+    }
+
+    /// <summary>Verifies a 65-character AckClientName fails and a 64-character one passes.</summary>
+    [Fact]
+    public void Validate_AckClientNameLengthBoundary()
+    {
+        GatewayOptionsValidator validator = new();
+        Assert.False(validator.Validate(
+            null, CloneWithAlarms(ValidOptions(), new AlarmsOptions { AckClientName = new string('a', 65) })).Succeeded);
+        Assert.True(validator.Validate(
+            null, CloneWithAlarms(ValidOptions(), new AlarmsOptions { AckClientName = new string('a', 64) })).Succeeded);
+    }
+
     /// <summary>Verifies the alarm poll cadence and per-fetch cap defaults pass validation.</summary>
     [Fact]
     public void Validate_Succeeds_WithDefaultAlarmPollCadenceAndFetchCap()

src/ZB.MOM.WW.MxGateway.Tests/Diagnostics/GatewayLogRedactorSeamTests.cs

@@ -84,4 +84,23 @@ public sealed class GatewayLogRedactorSeamTests
 
         Assert.False(props.ContainsKey("CommandValue"));
     }
+
+    /// <summary>Any property whose name contains "password" is masked regardless of casing.</summary>
+    [Fact]
+    public void Redact_PasswordNamedProperty_IsRedacted()
+    {
+        GatewayLogRedactorSeam redactor = new();
+        Dictionary<string, object?> props = new()
+        {
+            ["OperatorPassword"] = "pw-secret",
+            ["verify_user_PASSWORD"] = "pw-secret",
+            ["OperatorUser"] = "alice",
+        };
+
+        redactor.Redact(props);
+
+        Assert.Equal("[redacted]", props["OperatorPassword"]);
+        Assert.Equal("[redacted]", props["verify_user_PASSWORD"]);
+        Assert.Equal("alice", props["OperatorUser"]);
+    }
 }

src/ZB.MOM.WW.MxGateway.Tests/Diagnostics/GatewayLogRedactorTests.cs

@@ -118,6 +118,8 @@ public sealed class GatewayLogRedactorTests
     [InlineData("AuthenticateUser")]
     [InlineData("WriteSecured")]
     [InlineData("WriteSecured2")]
+    [InlineData("AcknowledgeAlarm")]
+    [InlineData("AcknowledgeAlarmByName")]
     public void IsCredentialBearingCommand_IdentifiesSensitiveMxAccessCommands(string commandMethod)
     {
         Assert.True(GatewayLogRedactor.IsCredentialBearingCommand(commandMethod));

src/ZB.MOM.WW.MxGateway.Tests/Gateway/Grpc/MxAccessGatewayServiceTests.cs

@@ -389,6 +389,22 @@ public sealed class MxAccessGatewayServiceTests
         Assert.Equal(StatusCode.InvalidArgument, exception.StatusCode);
     }
 
+    /// <summary>Verifies AcknowledgeAlarm rejects an empty operator_user.</summary>
+    /// <returns>A task that represents the asynchronous operation.</returns>
+    [Fact]
+    public async Task AcknowledgeAlarm_WithMissingOperatorUser_ThrowsInvalidArgument()
+    {
+        MxAccessGatewayService service = CreateService(new FakeSessionManager());
+
+        RpcException exception = await Assert.ThrowsAsync<RpcException>(
+            async () => await service.AcknowledgeAlarm(
+                new AcknowledgeAlarmRequest { AlarmFullReference = "Galaxy!Area.Tag" },
+                new TestServerCallContext()));
+
+        Assert.Equal(StatusCode.InvalidArgument, exception.StatusCode);
+        Assert.Contains("operator_user", exception.Status.Detail, StringComparison.Ordinal);
+    }
+
     /// <summary>Verifies AcknowledgeAlarm delegates a valid request to the alarm service.</summary>
     /// <returns>A task that represents the asynchronous operation.</returns>
     [Fact]

src/ZB.MOM.WW.MxGateway.Tests/Gateway/Workers/WorkerProcessLauncherTests.cs

@@ -22,6 +22,7 @@ public sealed class WorkerProcessLauncherTests
         WorkerLaunchEnvironment.EventQueueCapacity,
         WorkerLaunchEnvironment.AlarmPollIntervalMilliseconds,
         WorkerLaunchEnvironment.MaxAlarmsPerFetch,
+        WorkerLaunchEnvironment.AlarmAckClientName,
     ];
 
     /// <summary>Verifies that a valid worker executable starts with correct bootstrap arguments and nonce environment variable.</summary>
@@ -75,6 +76,10 @@ public sealed class WorkerProcessLauncherTests
             "1024",
             processFactory.LastStartInfo.Environment[
                 WorkerLaunchEnvironment.MaxAlarmsPerFetch]);
+        Assert.Equal(
+            "MxGateway",
+            processFactory.LastStartInfo.Environment[
+                WorkerLaunchEnvironment.AlarmAckClientName]);
         Assert.All(
             RequiredLaunchEnvironmentVariables,
             variable => Assert.True(
@@ -86,6 +91,28 @@ public sealed class WorkerProcessLauncherTests
         Assert.Equal(0, metrics.GetSnapshot().WorkersRunning);
     }
 
+    /// <summary>Verifies a configured AckClientName reaches the worker environment.</summary>
+    /// <returns>A task that represents the asynchronous operation.</returns>
+    [Fact]
+    public async Task LaunchAsync_WithConfiguredAckClientName_ExportsItToTheWorkerEnvironment()
+    {
+        using TestDirectory directory = TestDirectory.Create();
+        string executablePath = directory.CreateWorkerExecutable(machine: 0x014c);
+        FakeWorkerProcessFactory processFactory = new(new FakeWorkerProcess(processId: 1234));
+        WorkerProcessLauncher launcher = CreateLauncher(
+            executablePath,
+            processFactory,
+            new SucceedingStartupProbe(),
+            ackClientName: "Plant-Ack");
+
+        using WorkerProcessHandle handle = await launcher.LaunchAsync(CreateRequest());
+
+        Assert.NotNull(processFactory.LastStartInfo);
+        Assert.Equal(
+            "Plant-Ack",
+            processFactory.LastStartInfo.Environment[WorkerLaunchEnvironment.AlarmAckClientName]);
+    }
+
     /// <summary>
     /// Verifies that a configured <see cref="WorkerOptions.EventQueueCapacity"/> — not the shipped
     /// default — is what reaches the worker, so the option is deployable without a worker rebuild.
@@ -265,10 +292,12 @@ public sealed class WorkerProcessLauncherTests
         int startupTimeoutSeconds = 30,
         int startupProbeRetryAttempts = 3,
         int startupProbeRetryDelayMilliseconds = 250,
-        int eventQueueCapacity = 10000)
+        int eventQueueCapacity = 10000,
+        string ackClientName = "MxGateway")
     {
         GatewayOptions options = new()
         {
+            Alarms = new AlarmsOptions { AckClientName = ackClientName },
             Worker = new WorkerOptions
             {
                 ExecutablePath = executablePath,

src/ZB.MOM.WW.MxGateway.Worker.Tests/Bootstrap/WorkerApplicationTests.cs

@@ -1,4 +1,5 @@
 using System;
+using System.Collections.Generic;
 using System.Threading;
 using System.Threading.Tasks;
 using ZB.MOM.WW.MxGateway.Contracts;
@@ -121,6 +122,49 @@ public sealed class WorkerApplicationTests
         Assert.Equal(typeof(InvalidOperationException).FullName, entry.Fields["exception_type"]);
     }
 
+    /// <summary>
+    ///     WRK-09: the AppDomain unhandled-exception hook logs the exception type
+    ///     (never its message) and exits with the dedicated exit code.
+    /// </summary>
+    [Fact]
+    public void HandleUnhandledException_LogsTypeOnlyAndExitsWithUnhandledExceptionCode()
+    {
+        MemoryWorkerLogger logger = new();
+        int? exitCode = null;
+
+        ZB.MOM.WW.MxGateway.Worker.WorkerApplication.HandleUnhandledException(
+            logger,
+            new InvalidOperationException("password=hunter2"),
+            isTerminating: true,
+            code => exitCode = code);
+
+        Assert.Equal((int)WorkerExitCode.UnhandledException, exitCode);
+        Assert.Equal(7, (int)WorkerExitCode.UnhandledException);
+        MemoryWorkerLogEntry entry = Assert.Single(logger.Entries);
+        Assert.Equal("Error", entry.Level);
+        Assert.Equal("WorkerUnhandledException", entry.EventName);
+        Assert.Equal(typeof(InvalidOperationException).FullName, entry.Fields["exception_type"]);
+        Assert.True(entry.Fields["is_terminating"] is true);
+        Assert.DoesNotContain(
+            entry.Fields.Values,
+            value => value?.ToString()?.Contains("hunter2") == true);
+    }
+
+    /// <summary>A failing logger must not prevent the defined exit.</summary>
+    [Fact]
+    public void HandleUnhandledException_WhenLoggerThrows_StillExits()
+    {
+        int? exitCode = null;
+
+        ZB.MOM.WW.MxGateway.Worker.WorkerApplication.HandleUnhandledException(
+            new ThrowingLogger(),
+            exceptionObject: null,
+            isTerminating: false,
+            code => exitCode = code);
+
+        Assert.Equal((int)WorkerExitCode.UnhandledException, exitCode);
+    }
+
     private static string[] ValidArgs(string? protocolVersion = null)
     {
         return
@@ -171,4 +215,15 @@ public sealed class WorkerApplicationTests
             throw _exception;
         }
     }
+
+    private sealed class ThrowingLogger : IWorkerLogger
+    {
+        /// <inheritdoc />
+        public void Information(string eventName, IReadOnlyDictionary<string, object?> fields) =>
+            throw new InvalidOperationException("logger failed");
+
+        /// <inheritdoc />
+        public void Error(string eventName, IReadOnlyDictionary<string, object?> fields) =>
+            throw new InvalidOperationException("logger failed");
+    }
 }

src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/AlarmAckWriterTests.cs

@@ -84,7 +84,9 @@ public sealed class AlarmAckWriterTests
     public void Acknowledge_Twice_ReusesItemHandle()
     {
         RecordingAckServer server = new RecordingAckServer { AuthenticateResult = 5 };
-        using AlarmAckWriter writer = new AlarmAckWriter(server, new MxAccessWriteCompletionCache(), () => { }, TimeSpan.FromMilliseconds(10), "MxGateway");
+        MxAccessWriteCompletionCache cache = new MxAccessWriteCompletionCache();
+        server.OnWrite = (handle, item) => cache.Record(handle, item, Rows(MxStatusCategory.Ok, 0));
+        using AlarmAckWriter writer = new AlarmAckWriter(server, cache, () => { }, TimeSpan.FromSeconds(1), "MxGateway");
 
         writer.Acknowledge("T.A.AckMsg", "one", Alice());
         writer.Acknowledge("T.A.AckMsg", "two", Alice());
@@ -114,7 +116,9 @@ public sealed class AlarmAckWriterTests
     public void Acknowledge_WhenCachedWriteThrows_RejectsAndReAddsNextTime()
     {
         RecordingAckServer server = new RecordingAckServer { AuthenticateResult = 5 };
-        using AlarmAckWriter writer = new AlarmAckWriter(server, new MxAccessWriteCompletionCache(), () => { }, TimeSpan.FromMilliseconds(10), "MxGateway");
+        MxAccessWriteCompletionCache cache = new MxAccessWriteCompletionCache();
+        server.OnWrite = (handle, item) => cache.Record(handle, item, Rows(MxStatusCategory.Ok, 0));
+        using AlarmAckWriter writer = new AlarmAckWriter(server, cache, () => { }, TimeSpan.FromSeconds(1), "MxGateway");
         writer.Acknowledge("T.A.AckMsg", "one", Alice());
         server.WriteException = new COMException("x", unchecked((int)0x80004005));
 
@@ -141,6 +145,265 @@ public sealed class AlarmAckWriterTests
         Assert.Contains("Unregister", server.Operations);
     }
 
+    [Fact]
+    public void Acknowledge_WhenAuthenticateUserDisconnected_TearsDownAndRejects()
+    {
+        RecordingAckServer server = new RecordingAckServer
+        {
+            AuthenticateResult = 5,
+            AuthenticateException = new COMException("gone pw-secret", unchecked((int)0x80010108)),
+        };
+        using AlarmAckWriter writer = new AlarmAckWriter(server, new MxAccessWriteCompletionCache(), () => { }, TimeSpan.FromMilliseconds(10), "MxGateway");
+
+        AlarmAckResult result = writer.Acknowledge("T.A.AckMsg", "one", Alice());
+
+        Assert.Equal(AlarmAckOutcome.WriteRejected, result.Outcome);
+        Assert.Equal(unchecked((int)0x80010108), result.NativeStatus);
+        Assert.Contains("AuthenticateUser", result.Diagnostic, StringComparison.Ordinal);
+        Assert.Contains("0x80010108", result.Diagnostic, StringComparison.Ordinal);
+        Assert.DoesNotContain("pw-secret", result.Diagnostic, StringComparison.Ordinal);
+        Assert.Contains("Unregister", server.Operations);
+        Assert.DoesNotContain(server.Operations, op => op.StartsWith("AddItem", StringComparison.Ordinal));
+    }
+
+    [Fact]
+    public void Acknowledge_WhenAuthenticateUserThrowsArgumentException_ReturnsAuthenticationFailed()
+    {
+        // Live MXAccess raises E_INVALIDARG for a bad user or wrong password, which the
+        // interop surfaces as ArgumentException rather than COMException.
+        RecordingAckServer server = new RecordingAckServer
+        {
+            AuthenticateResult = 5,
+            AuthenticateException = new ArgumentException("Value does not fall within the expected range. pw-secret"),
+        };
+        using AlarmAckWriter writer = new AlarmAckWriter(server, new MxAccessWriteCompletionCache(), () => { }, TimeSpan.FromMilliseconds(10), "MxGateway");
+
+        AlarmAckResult result = writer.Acknowledge("T.A.AckMsg", "one", Alice());
+
+        Assert.Equal(AlarmAckOutcome.AuthenticationFailed, result.Outcome);
+        Assert.Equal(unchecked((int)0x80070057), result.NativeStatus);
+        Assert.DoesNotContain("pw-secret", result.Diagnostic, StringComparison.Ordinal);
+        Assert.DoesNotContain(server.Operations, op => op.StartsWith("Write", StringComparison.Ordinal));
+        Assert.DoesNotContain("Unregister", server.Operations);
+    }
+
+    [Fact]
+    public void Acknowledge_WhenAuthenticateUserThrowsArgumentNullException_Propagates()
+    {
+        // A programming error is a bug, not an operator authentication failure.
+        RecordingAckServer server = new RecordingAckServer
+        {
+            AuthenticateResult = 5,
+            AuthenticateException = new ArgumentNullException("verifyUser"),
+        };
+        using AlarmAckWriter writer = new AlarmAckWriter(server, new MxAccessWriteCompletionCache(), () => { }, TimeSpan.FromMilliseconds(10), "MxGateway");
+
+        Assert.Throws<ArgumentNullException>(() => writer.Acknowledge("T.A.AckMsg", "one", Alice()));
+    }
+
+    [Fact]
+    public void Acknowledge_WhenOwnCodeOutsideComCallThrowsArgumentException_Propagates()
+    {
+        // The pump runs between Advise and Write. An ArgumentException from it carries
+        // the same default HRESULT (0x80070057) as an interop E_INVALIDARG, so only
+        // catching at the COM call itself keeps it from being reported as WriteRejected.
+        RecordingAckServer server = new RecordingAckServer { AuthenticateResult = 5 };
+        using AlarmAckWriter writer = new AlarmAckWriter(server, new MxAccessWriteCompletionCache(), () => throw new ArgumentException("bug"), TimeSpan.FromMilliseconds(10), "MxGateway");
+
+        ArgumentException thrown = Assert.Throws<ArgumentException>(() => writer.Acknowledge("T.A.AckMsg", "one", Alice()));
+        Assert.Equal("bug", thrown.Message);
+        Assert.DoesNotContain(server.Operations, op => op.StartsWith("Write", StringComparison.Ordinal));
+    }
+
+    [Fact]
+    public void Acknowledge_WhenWriteThrowsArgumentOutOfRangeException_Propagates()
+    {
+        RecordingAckServer server = new RecordingAckServer { AuthenticateResult = 5, WriteException = new ArgumentOutOfRangeException("value") };
+        using AlarmAckWriter writer = new AlarmAckWriter(server, new MxAccessWriteCompletionCache(), () => { }, TimeSpan.FromMilliseconds(10), "MxGateway");
+
+        Assert.Throws<ArgumentOutOfRangeException>(() => writer.Acknowledge("T.A.AckMsg", "one", Alice()));
+    }
+
+    [Fact]
+    public void Acknowledge_WhenOperatorUserNameIsNull_ThrowsBeforeAuthenticating()
+    {
+        RecordingAckServer server = new RecordingAckServer { AuthenticateResult = 5 };
+        using AlarmAckWriter writer = new AlarmAckWriter(server, new MxAccessWriteCompletionCache(), () => { }, TimeSpan.FromMilliseconds(10), "MxGateway");
+
+        Assert.Throws<ArgumentException>(() => writer.Acknowledge("T.A.AckMsg", "one", new AlarmAckOperator { UserName = null! }));
+        Assert.DoesNotContain(server.Operations, op => op.StartsWith("AuthenticateUser", StringComparison.Ordinal));
+    }
+
+    [Fact]
+    public void Acknowledge_WhenWriteThrowsArgumentException_RejectsAndDropsItem()
+    {
+        RecordingAckServer server = new RecordingAckServer { AuthenticateResult = 5, WriteException = new ArgumentException("bad") };
+        using AlarmAckWriter writer = new AlarmAckWriter(server, new MxAccessWriteCompletionCache(), () => { }, TimeSpan.FromMilliseconds(10), "MxGateway");
+
+        AlarmAckResult result = writer.Acknowledge("T.A.AckMsg", "one", Alice());
+
+        Assert.Equal(AlarmAckOutcome.WriteRejected, result.Outcome);
+        Assert.Equal(unchecked((int)0x80070057), result.NativeStatus);
+        Assert.Contains("Write", result.Diagnostic, StringComparison.Ordinal);
+        Assert.Equal(new[] { 100 }, server.RemovedItemHandles);
+        Assert.DoesNotContain("Unregister", server.Operations);
+    }
+
+    [Fact]
+    public void Acknowledge_PastCacheCap_EvictsLeastRecentlyUsedItem()
+    {
+        RecordingAckServer server = new RecordingAckServer { AuthenticateResult = 5 };
+        using AlarmAckWriter writer = new AlarmAckWriter(server, new MxAccessWriteCompletionCache(), () => { }, TimeSpan.FromMilliseconds(1), "MxGateway", maxCachedItems: 2);
+
+        writer.Acknowledge("A.X.AckMsg", "1", Alice()); // handle 100
+        writer.Acknowledge("B.X.AckMsg", "2", Alice()); // handle 101
+        writer.Acknowledge("C.X.AckMsg", "3", Alice()); // handle 102, evicts A (100)
+
+        Assert.Equal(new[] { 100 }, server.RemovedItemHandles);
+        Assert.Equal(3, server.Operations.Count(op => op.StartsWith("AddItem", StringComparison.Ordinal)));
+    }
+
+    [Fact]
+    public void Acknowledge_ReusedItem_IsMostRecentlyUsedForEviction()
+    {
+        RecordingAckServer server = new RecordingAckServer { AuthenticateResult = 5 };
+        MxAccessWriteCompletionCache cache = new MxAccessWriteCompletionCache();
+        server.OnWrite = (handle, item) => cache.Record(handle, item, Rows(MxStatusCategory.Ok, 0));
+        using AlarmAckWriter writer = new AlarmAckWriter(server, cache, () => { }, TimeSpan.FromSeconds(1), "MxGateway", maxCachedItems: 2);
+
+        writer.Acknowledge("A.X.AckMsg", "1", Alice()); // handle 100
+        writer.Acknowledge("B.X.AckMsg", "2", Alice()); // handle 101
+        writer.Acknowledge("A.X.AckMsg", "3", Alice()); // reuse 100, now most recent
+        writer.Acknowledge("C.X.AckMsg", "4", Alice()); // handle 102, evicts B (101)
+
+        Assert.Equal(new[] { 101 }, server.RemovedItemHandles);
+        Assert.Equal(3, server.Operations.Count(op => op.StartsWith("AddItem", StringComparison.Ordinal)));
+        Assert.Contains("UnAdvise", server.Operations);
+    }
+
+    [Fact]
+    public void Acknowledge_AfterUnconfirmed_UsesFreshHandleSoStaleCompletionIsNotCredited()
+    {
+        RecordingAckServer server = new RecordingAckServer { AuthenticateResult = 5 };
+        MxAccessWriteCompletionCache cache = new MxAccessWriteCompletionCache();
+        using AlarmAckWriter writer = new AlarmAckWriter(server, cache, () => { }, TimeSpan.FromMilliseconds(50), "MxGateway");
+
+        AlarmAckResult first = writer.Acknowledge("T.A.AckMsg", "one", Alice()); // handle 100, no completion
+        // The first write's late OnWriteComplete (handle 100) lands while the
+        // second write is in flight; the second write's own completion never comes.
+        server.OnWrite = (handle, item) => cache.Record(handle, 100, Rows(MxStatusCategory.Ok, 0));
+        AlarmAckResult second = writer.Acknowledge("T.A.AckMsg", "two", Alice());
+
+        Assert.Equal(AlarmAckOutcome.Unconfirmed, first.Outcome);
+        Assert.Equal(AlarmAckOutcome.Unconfirmed, second.Outcome);
+        Assert.Contains(100, server.RemovedItemHandles);
+        Assert.Equal(2, server.Operations.Count(op => op.StartsWith("AddItem", StringComparison.Ordinal)));
+        Assert.Equal(new[] { 100, 101 }, server.WrittenItemHandles);
+    }
+
+    [Fact]
+    public void Acknowledge_AfterConfirmed_ReusesHandle()
+    {
+        RecordingAckServer server = new RecordingAckServer { AuthenticateResult = 5 };
+        MxAccessWriteCompletionCache cache = new MxAccessWriteCompletionCache();
+        server.OnWrite = (handle, item) => cache.Record(handle, item, Rows(MxStatusCategory.Ok, 0));
+        using AlarmAckWriter writer = new AlarmAckWriter(server, cache, () => { }, TimeSpan.FromSeconds(1), "MxGateway");
+
+        writer.Acknowledge("T.A.AckMsg", "one", Alice());
+        writer.Acknowledge("T.A.AckMsg", "two", Alice());
+
+        Assert.Equal(new[] { 100, 100 }, server.WrittenItemHandles);
+        Assert.Empty(server.RemovedItemHandles);
+    }
+
+    [Fact]
+    public void Acknowledge_AfterDirtyReAdd_ForgetsDroppedHandleCompletion()
+    {
+        RecordingAckServer server = new RecordingAckServer { AuthenticateResult = 5 };
+        MxAccessWriteCompletionCache cache = new MxAccessWriteCompletionCache();
+        using AlarmAckWriter writer = new AlarmAckWriter(server, cache, () => { }, TimeSpan.FromMilliseconds(20), "MxGateway");
+
+        writer.Acknowledge("T.A.AckMsg", "one", Alice()); // handle 100, Unconfirmed
+        cache.Record(7, 100, Rows(MxStatusCategory.Ok, 0)); // its late completion
+        Assert.Equal(1, cache.Count);
+        server.OnWrite = (handle, item) => cache.Record(handle, item, Rows(MxStatusCategory.Ok, 0));
+
+        AlarmAckResult second = writer.Acknowledge("T.A.AckMsg", "two", Alice()); // drops 100, adds 101
+
+        Assert.Equal(AlarmAckOutcome.Applied, second.Outcome);
+        Assert.Equal(1, cache.Count);
+        Assert.Equal(0UL, cache.CurrentVersion(7, 100));
+        Assert.Equal(1UL, cache.CurrentVersion(7, 101));
+    }
+
+    [Fact]
+    public void LateCompletionForDroppedHandle_DoesNotRecreateCacheEntry()
+    {
+        RecordingAckServer server = new RecordingAckServer { AuthenticateResult = 5 };
+        MxAccessWriteCompletionCache cache = new MxAccessWriteCompletionCache();
+        using AlarmAckWriter writer = new AlarmAckWriter(server, cache, () => { }, TimeSpan.FromMilliseconds(20), "MxGateway");
+        writer.Acknowledge("T.A.AckMsg", "one", Alice()); // handle 100, Unconfirmed
+        server.OnWrite = (handle, item) => writer.RecordWriteCompletion(handle, item, Rows(MxStatusCategory.Ok, 0));
+        writer.Acknowledge("T.A.AckMsg", "two", Alice()); // drops 100, adds 101, applied
+
+        writer.RecordWriteCompletion(7, 100, Rows(MxStatusCategory.Ok, 0)); // 100's late completion
+        writer.RecordWriteCompletion(8, 101, Rows(MxStatusCategory.Ok, 0)); // another registration
+
+        Assert.Equal(1, cache.Count);
+        Assert.Equal(0UL, cache.CurrentVersion(7, 100));
+        Assert.Equal(1UL, cache.CurrentVersion(7, 101));
+    }
+
+    [Fact]
+    public void Acknowledge_PastCacheCap_ForgetsEvictedHandleCompletion()
+    {
+        RecordingAckServer server = new RecordingAckServer { AuthenticateResult = 5 };
+        MxAccessWriteCompletionCache cache = new MxAccessWriteCompletionCache();
+        server.OnWrite = (handle, item) => cache.Record(handle, item, Rows(MxStatusCategory.Ok, 0));
+        using AlarmAckWriter writer = new AlarmAckWriter(server, cache, () => { }, TimeSpan.FromSeconds(1), "MxGateway", maxCachedItems: 2);
+
+        writer.Acknowledge("A.X.AckMsg", "1", Alice()); // handle 100
+        writer.Acknowledge("B.X.AckMsg", "2", Alice()); // handle 101
+        Assert.Equal(2, cache.Count);
+        writer.Acknowledge("C.X.AckMsg", "3", Alice()); // handle 102, evicts A (100)
+
+        Assert.Equal(2, cache.Count);
+        Assert.Equal(0UL, cache.CurrentVersion(7, 100));
+    }
+
+    [Fact]
+    public void Dispose_ForgetsServerCompletions()
+    {
+        RecordingAckServer server = new RecordingAckServer { AuthenticateResult = 5 };
+        MxAccessWriteCompletionCache cache = new MxAccessWriteCompletionCache();
+        server.OnWrite = (handle, item) => cache.Record(handle, item, Rows(MxStatusCategory.Ok, 0));
+        AlarmAckWriter writer = new AlarmAckWriter(server, cache, () => { }, TimeSpan.FromSeconds(1), "MxGateway");
+        writer.Acknowledge("A.X.AckMsg", "1", Alice());
+        writer.Acknowledge("B.X.AckMsg", "2", Alice());
+        cache.Record(8, 100, Rows(MxStatusCategory.Ok, 0)); // another server's entry survives
+
+        writer.Dispose();
+
+        Assert.Equal(1, cache.Count);
+        Assert.Equal(1UL, cache.CurrentVersion(8, 100));
+    }
+
+    [Fact]
+    public void Acknowledge_WhenWriteDisconnected_ForgetsServerCompletions()
+    {
+        RecordingAckServer server = new RecordingAckServer { AuthenticateResult = 5 };
+        MxAccessWriteCompletionCache cache = new MxAccessWriteCompletionCache();
+        server.OnWrite = (handle, item) => cache.Record(handle, item, Rows(MxStatusCategory.Ok, 0));
+        using AlarmAckWriter writer = new AlarmAckWriter(server, cache, () => { }, TimeSpan.FromSeconds(1), "MxGateway");
+        writer.Acknowledge("A.X.AckMsg", "1", Alice()); // handle 100 recorded
+        server.WriteException = new COMException("x", unchecked((int)0x80010108));
+
+        writer.Acknowledge("A.X.AckMsg", "2", Alice());
+
+        Assert.Contains("Unregister", server.Operations);
+        Assert.Equal(0, cache.Count);
+    }
+
     [Fact]
     public void Acknowledge_WhenNonOkCompletionHasNoDetail_ReturnsMinusOne()
     {
@@ -180,6 +443,10 @@ internal sealed class RecordingAckServer : IMxAccessServer
 
     public Action<int, int>? OnWrite { get; set; }
 
+    public List<int> RemovedItemHandles { get; } = new();
+
+    public List<int> WrittenItemHandles { get; } = new();
+
     public int Register(string clientName)
     {
         Operations.Add("Register:" + clientName);
@@ -196,7 +463,11 @@ internal sealed class RecordingAckServer : IMxAccessServer
 
     public int AddItem2(int serverHandle, string itemDefinition, string itemContext) => AddItem(serverHandle, itemDefinition);
 
-    public void RemoveItem(int serverHandle, int itemHandle) => Operations.Add("RemoveItem");
+    public void RemoveItem(int serverHandle, int itemHandle)
+    {
+        Operations.Add("RemoveItem");
+        RemovedItemHandles.Add(itemHandle);
+    }
 
     public void Advise(int serverHandle, int itemHandle)
     {
@@ -216,6 +487,7 @@ internal sealed class RecordingAckServer : IMxAccessServer
     public void Write(int serverHandle, int itemHandle, object? value, int userId)
     {
         Operations.Add("Write:" + value + ":" + userId);
+        WrittenItemHandles.Add(itemHandle);
         if (WriteException is not null)
         {
             throw WriteException;

src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/AlarmCommandHandlerTests.cs

@@ -499,6 +499,40 @@ public sealed class AlarmCommandHandlerTests
         Assert.Equal(AlarmProviderMode.Subtag, evt.OnAlarmProviderModeChanged.Mode);
     }
 
+    /// <summary>
+    ///     In auto mode the primary consumer is built before the standby. When
+    ///     building the standby throws, the primary is owned by nothing — the
+    ///     failed Subscribe never reaches the dispatcher that would dispose it —
+    ///     so the handler must dispose it rather than leak its COM object, and
+    ///     the original failure must still propagate.
+    /// </summary>
+    [Fact]
+    public void Subscribe_AutoModeWhenStandbyBuildThrows_DisposesPrimaryAndRethrows()
+    {
+        FakeConsumer primary = new FakeConsumer();
+        InvalidOperationException standbyFailure = new InvalidOperationException("standby build failed");
+        AlarmCommandHandler handler = new AlarmCommandHandler(
+            new MxAccessEventQueue(),
+            () => primary,
+            threadAffinityCheck: null,
+            comFactory: null,
+            standbyFactory: _ => throw standbyFailure);
+
+        SubscribeAlarmsCommand command = new SubscribeAlarmsCommand
+        {
+            SubscriptionExpression = @"\\HOST\Galaxy!Area",
+            ForcedMode = AlarmProviderMode.Unspecified,
+        };
+        command.WatchList.Add(NewTarget("Galaxy!Area.Tank01.Level.HiHi"));
+
+        InvalidOperationException thrown = Assert.Throws<InvalidOperationException>(
+            () => handler.Subscribe(command, "s1"));
+
+        Assert.Same(standbyFailure, thrown);
+        Assert.True(primary.Disposed, "The primary consumer built before the failing standby must be disposed.");
+        Assert.False(handler.IsSubscribed);
+    }
+
     /// <summary>An explicitly fallback-capable auto subscription builds a replaceable standby even when discovery is initially empty.</summary>
     [Fact]
     public void Subscribe_AutoModeWithEmptyWatchListAndFallbackEnabled_BuildsReplaceableStandby()

src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/AlarmRecordTransitionMapperTests.cs

@@ -102,6 +102,22 @@ public sealed class AlarmRecordTransitionMapperTests
         Assert.Equal(expected, AlarmRecordTransitionMapper.MapTransition(previous, current));
     }
 
+    /// <summary>
+    ///     RETRIGGER is reserved and never synthesized: no state pair — including
+    ///     cleared-to-active, the re-alarm shape — maps to it.
+    /// </summary>
+    [Fact]
+    public void MapTransition_ForEveryStatePair_NeverProducesRetrigger()
+    {
+        foreach (MxAlarmStateKind previous in System.Enum.GetValues(typeof(MxAlarmStateKind)))
+        {
+            foreach (MxAlarmStateKind current in System.Enum.GetValues(typeof(MxAlarmStateKind)))
+            {
+                Assert.NotEqual(AlarmTransitionKind.Retrigger, AlarmRecordTransitionMapper.MapTransition(previous, current));
+            }
+        }
+    }
+
     /// <summary>Verifies that valid XML fields assemble into correct UTC timestamp.</summary>
     [Fact]
     public void ParseTransitionTimestampUtc_WithValidXmlFields_AssemblesUtc()

src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/MxAccessSessionTests.cs

@@ -0,0 +1,591 @@
+using System;
+using System.Collections.Generic;
+using System.Diagnostics;
+using System.Linq;
+using Google.Protobuf.Collections;
+using Google.Protobuf.WellKnownTypes;
+using ZB.MOM.WW.MxGateway.Contracts.Proto;
+using ZB.MOM.WW.MxGateway.Worker.MxAccess;
+using ZB.MOM.WW.MxGateway.Worker.Tests.TestSupport;
+
+namespace ZB.MOM.WW.MxGateway.Worker.Tests.MxAccess;
+
+/// <summary>
+///     Direct tests of <see cref="MxAccessSession"/> handle-lifetime cache
+///     eviction and the phased ReadBulk snapshot path, driven through
+///     <see cref="MxAccessSession.CreateForTesting"/> with a recording fake
+///     server. Nothing here needs an STA: the session methods are plain calls,
+///     and ReadBulk's pump step is a test delegate that stands in for the COM
+///     OnDataChange dispatch by writing the value cache directly.
+/// </summary>
+public sealed class MxAccessSessionTests
+{
+    private const string ReadBulkTimeoutText = "timed out";
+
+    /// <summary>
+    ///     RemoveItem must evict the handle pair's OnWriteComplete completion
+    ///     entry as well as its last-value entry; otherwise an
+    ///     AddItem -> Write -> RemoveItem churn grows the completion cache by one
+    ///     entry per cycle for the life of the worker.
+    /// </summary>
+    [Fact]
+    public void RemoveItem_EvictsValueAndWriteCompletionEntriesForTheHandle()
+    {
+        RecordingMxAccessServer server = new();
+        CompletionCacheSink sink = new();
+        MxAccessValueCache valueCache = new();
+        using MxAccessSession session = MxAccessSession.CreateForTesting(server, sink, valueCache: valueCache);
+
+        int serverHandle = session.Register("client");
+        int keep = session.AddItem(serverHandle, "Tag.Keep");
+        int drop = session.AddItem(serverHandle, "Tag.Drop");
+        RecordCompletion(sink.WriteCompletionCache, serverHandle, keep);
+        RecordCompletion(sink.WriteCompletionCache, serverHandle, drop);
+        valueCache.Set(serverHandle, keep, BuildDataChange(serverHandle, keep, 1));
+        valueCache.Set(serverHandle, drop, BuildDataChange(serverHandle, drop, 2));
+
+        session.RemoveItem(serverHandle, drop);
+
+        Assert.Equal(1, sink.WriteCompletionCache.Count);
+        Assert.Equal(0UL, sink.WriteCompletionCache.CurrentVersion(serverHandle, drop));
+        Assert.Equal(1UL, sink.WriteCompletionCache.CurrentVersion(serverHandle, keep));
+        Assert.False(valueCache.TryGet(serverHandle, drop, out _));
+        Assert.True(valueCache.TryGet(serverHandle, keep, out _));
+    }
+
+    /// <summary>
+    ///     A long-lived writer that keeps adding, writing, and removing items
+    ///     must leave the completion cache empty — the leak this fix closes.
+    /// </summary>
+    [Fact]
+    public void AddItemWriteRemoveItemChurn_DoesNotGrowWriteCompletionCache()
+    {
+        RecordingMxAccessServer server = new();
+        CompletionCacheSink sink = new();
+        using MxAccessSession session = MxAccessSession.CreateForTesting(server, sink);
+        int serverHandle = session.Register("client");
+
+        for (int cycle = 0; cycle < 50; cycle++)
+        {
+            int itemHandle = session.AddItem(serverHandle, "Tag.Churn");
+            session.Write(serverHandle, itemHandle, cycle, userId: 0);
+
+            // Stand in for the sink's OnWriteComplete post-publish hook.
+            RecordCompletion(sink.WriteCompletionCache, serverHandle, itemHandle);
+            session.RemoveItem(serverHandle, itemHandle);
+        }
+
+        Assert.Equal(0, sink.WriteCompletionCache.Count);
+    }
+
+    /// <summary>
+    ///     Unregister drops every item under the server in one MXAccess call,
+    ///     so the per-item eviction never runs for them: both caches must be
+    ///     evicted by server handle, and other servers' entries kept.
+    /// </summary>
+    [Fact]
+    public void Unregister_EvictsValueAndWriteCompletionEntriesForThatServerOnly()
+    {
+        RecordingMxAccessServer server = new();
+        CompletionCacheSink sink = new();
+        MxAccessValueCache valueCache = new();
+        using MxAccessSession session = MxAccessSession.CreateForTesting(server, sink, valueCache: valueCache);
+
+        int doomed = session.Register("doomed");
+        int survivor = session.Register("survivor");
+        int doomedItemA = session.AddItem(doomed, "Tag.A");
+        int doomedItemB = session.AddItem(doomed, "Tag.B");
+        int survivorItem = session.AddItem(survivor, "Tag.C");
+        RecordCompletion(sink.WriteCompletionCache, doomed, doomedItemA);
+        RecordCompletion(sink.WriteCompletionCache, doomed, doomedItemB);
+        RecordCompletion(sink.WriteCompletionCache, survivor, survivorItem);
+        valueCache.Set(doomed, doomedItemA, BuildDataChange(doomed, doomedItemA, 1));
+        valueCache.Set(doomed, doomedItemB, BuildDataChange(doomed, doomedItemB, 2));
+        valueCache.Set(survivor, survivorItem, BuildDataChange(survivor, survivorItem, 3));
+
+        session.Unregister(doomed);
+
+        Assert.Equal(1, sink.WriteCompletionCache.Count);
+        Assert.Equal(1UL, sink.WriteCompletionCache.CurrentVersion(survivor, survivorItem));
+        Assert.Equal(1, valueCache.Count);
+        Assert.True(valueCache.TryGet(survivor, survivorItem, out _));
+        Assert.False(valueCache.TryGet(doomed, doomedItemA, out _));
+        Assert.False(valueCache.TryGet(doomed, doomedItemB, out _));
+    }
+
+    /// <summary>
+    ///     N uncached tags that never deliver an OnDataChange must cost one
+    ///     shared timeout, not N x timeout, while each still reports its own
+    ///     timed-out failure with its own item handle.
+    /// </summary>
+    [Fact]
+    public void ReadBulk_UncachedTagsThatNeverDeliver_WaitOneSharedTimeoutAndEachTimesOut()
+    {
+        RecordingMxAccessServer server = new();
+        using MxAccessSession session = MxAccessSession.CreateForTesting(server, new NoopEventSink());
+        int serverHandle = session.Register("client");
+        string[] tags = { "Tag.1", "Tag.2", "Tag.3", "Tag.4", "Tag.5" };
+        TimeSpan timeout = TimeSpan.FromMilliseconds(300);
+
+        Stopwatch stopwatch = Stopwatch.StartNew();
+        IReadOnlyList<BulkReadResult> results = session.ReadBulk(serverHandle, tags, timeout, () => { });
+        stopwatch.Stop();
+
+        // Serial would be >= 5 x 300 ms = 1500 ms. Allow generous slack for a
+        // loaded CI host while still failing the serial shape.
+        Assert.True(
+            stopwatch.Elapsed < TimeSpan.FromMilliseconds(1200),
+            $"ReadBulk took {stopwatch.Elapsed.TotalMilliseconds:F0} ms; expected one shared {timeout.TotalMilliseconds:F0} ms wait.");
+        Assert.True(stopwatch.Elapsed >= TimeSpan.FromMilliseconds(250));
+
+        Assert.Equal(tags.Length, results.Count);
+        HashSet<int> handles = new();
+        for (int index = 0; index < tags.Length; index++)
+        {
+            BulkReadResult result = results[index];
+            Assert.Equal(tags[index], result.TagAddress);
+            Assert.False(result.WasSuccessful);
+            Assert.False(result.WasCached);
+            Assert.Equal(serverHandle, result.ServerHandle);
+            Assert.Equal(server.HandleForTag(tags[index]), result.ItemHandle);
+            Assert.Equal(
+                $"ReadBulk timed out after {timeout.TotalMilliseconds:F0} ms waiting for first OnDataChange.",
+                result.ErrorMessage);
+            Assert.True(handles.Add(result.ItemHandle));
+        }
+
+        // Every snapshot handle is still torn down.
+        foreach (int handle in handles)
+        {
+            Assert.Contains($"UnAdvise:{serverHandle}:{handle}", server.Operations);
+            Assert.Contains($"RemoveItem:{serverHandle}:{handle}", server.Operations);
+        }
+
+        Assert.Empty(session.HandleRegistry.ItemHandles);
+    }
+
+    /// <summary>
+    ///     The phased lifecycle: every uncached tag is added and advised before
+    ///     the wait, and none is torn down until the wait is over.
+    /// </summary>
+    [Fact]
+    public void ReadBulk_AddsAndAdvisesEverySnapshotTagBeforeTearingAnyDown()
+    {
+        RecordingMxAccessServer server = new();
+        using MxAccessSession session = MxAccessSession.CreateForTesting(server, new NoopEventSink());
+        int serverHandle = session.Register("client");
+
+        session.ReadBulk(serverHandle, new[] { "Tag.A", "Tag.B", "Tag.C" }, TimeSpan.FromMilliseconds(50), () => { });
+
+        List<string> lifecycle = server.Operations.Where(op => !op.StartsWith("Register", StringComparison.Ordinal)).ToList();
+        int lastAdvise = lifecycle.FindLastIndex(op => op.StartsWith("Advise:", StringComparison.Ordinal));
+        int firstUnAdvise = lifecycle.FindIndex(op => op.StartsWith("UnAdvise:", StringComparison.Ordinal));
+        int firstRemove = lifecycle.FindIndex(op => op.StartsWith("RemoveItem:", StringComparison.Ordinal));
+        Assert.Equal(3, lifecycle.Count(op => op.StartsWith("AddItem:", StringComparison.Ordinal)));
+        Assert.Equal(3, lifecycle.Count(op => op.StartsWith("Advise:", StringComparison.Ordinal)));
+        Assert.True(lastAdvise < firstUnAdvise, string.Join(",", lifecycle));
+        Assert.True(lastAdvise < firstRemove, string.Join(",", lifecycle));
+    }
+
+    /// <summary>
+    ///     Every per-tag outcome keeps its exact pre-batching shape and request
+    ///     order in one mixed request: blank address, cached-and-advised fast
+    ///     path, a tag that delivers, an AddItem failure, an Advise failure, and
+    ///     a tag that never delivers. A delivering tag returns as soon as its
+    ///     value lands rather than waiting out the timeout for the others.
+    /// </summary>
+    [Fact]
+    public void ReadBulk_MixedOutcomes_PreservesPerTagResultsAndOrder()
+    {
+        RecordingMxAccessServer server = new();
+        server.FailAddItemFor.Add("Tag.AddFails");
+        server.FailAdviseFor.Add("Tag.AdviseFails");
+        MxAccessValueCache valueCache = new();
+        using MxAccessSession session = MxAccessSession.CreateForTesting(server, new NoopEventSink(), valueCache: valueCache);
+        int serverHandle = session.Register("client");
+
+        // Caller-owned advised tag with a cached value: served from the cache.
+        int cachedHandle = session.AddItem(serverHandle, "Tag.Cached");
+        session.Advise(serverHandle, cachedHandle);
+        valueCache.Set(serverHandle, cachedHandle, BuildDataChange(serverHandle, cachedHandle, 11));
+        int operationsBeforeRead = server.Operations.Count;
+
+        // Pump step: deliver Tag.Delivers' first OnDataChange once advised.
+        Action pumpStep = () =>
+        {
+            if (server.TryGetAdvisedHandle("Tag.Delivers", out int handle)
+                && !valueCache.TryGet(serverHandle, handle, out _))
+            {
+                valueCache.Set(serverHandle, handle, BuildDataChange(serverHandle, handle, 42));
+            }
+        };
+
+        string[] tags = { " ", "Tag.Cached", "Tag.Delivers", "Tag.AddFails", "Tag.AdviseFails", "Tag.Silent" };
+        IReadOnlyList<BulkReadResult> results = session.ReadBulk(serverHandle, tags, TimeSpan.FromMilliseconds(150), pumpStep);
+
+        Assert.Equal(tags.Length, results.Count);
+        for (int index = 0; index < tags.Length; index++)
+        {
+            Assert.Equal(tags[index], results[index].TagAddress);
+        }
+
+        Assert.False(results[0].WasSuccessful);
+        Assert.Equal(0, results[0].ItemHandle);
+        Assert.Equal("Tag address is required.", results[0].ErrorMessage);
+
+        Assert.True(results[1].WasSuccessful);
+        Assert.True(results[1].WasCached);
+        Assert.Equal(cachedHandle, results[1].ItemHandle);
+        Assert.Equal(11, results[1].Value.Int32Value);
+
+        Assert.True(results[2].WasSuccessful);
+        Assert.False(results[2].WasCached);
+        Assert.Equal(server.HandleForTag("Tag.Delivers"), results[2].ItemHandle);
+        Assert.Equal(42, results[2].Value.Int32Value);
+        Assert.Equal(192, results[2].Quality);
+
+        Assert.False(results[3].WasSuccessful);
+        Assert.Equal(0, results[3].ItemHandle);
+        Assert.Equal("AddItem failed for Tag.AddFails", results[3].ErrorMessage);
+
+        Assert.False(results[4].WasSuccessful);
+        Assert.Equal(server.HandleForTag("Tag.AdviseFails"), results[4].ItemHandle);
+        Assert.Equal("Advise failed for Tag.AdviseFails", results[4].ErrorMessage);
+
+        Assert.False(results[5].WasSuccessful);
+        Assert.Equal(server.HandleForTag("Tag.Silent"), results[5].ItemHandle);
+        Assert.Contains(ReadBulkTimeoutText, results[5].ErrorMessage, StringComparison.OrdinalIgnoreCase);
+
+        // The caller's subscription is untouched; the snapshot handles are gone
+        // (the Advise-failed one removed without an UnAdvise).
+        List<string> readOperations = server.Operations.Skip(operationsBeforeRead).ToList();
+        Assert.DoesNotContain($"UnAdvise:{serverHandle}:{cachedHandle}", readOperations);
+        Assert.DoesNotContain($"RemoveItem:{serverHandle}:{cachedHandle}", readOperations);
+        Assert.DoesNotContain($"UnAdvise:{serverHandle}:{server.HandleForTag("Tag.AdviseFails")}", readOperations);
+        Assert.Contains($"RemoveItem:{serverHandle}:{server.HandleForTag("Tag.AdviseFails")}", readOperations);
+        Assert.Contains($"UnAdvise:{serverHandle}:{server.HandleForTag("Tag.Delivers")}", readOperations);
+        Assert.Contains($"RemoveItem:{serverHandle}:{server.HandleForTag("Tag.Silent")}", readOperations);
+        RegisteredItemHandle remaining = Assert.Single(session.HandleRegistry.ItemHandles);
+        Assert.Equal(cachedHandle, remaining.ItemHandle);
+    }
+
+    /// <summary>
+    ///     When every snapshot tag delivers, the wait ends as soon as the last
+    ///     value lands instead of running out the deadline.
+    /// </summary>
+    [Fact]
+    public void ReadBulk_AllTagsDeliver_ReturnsWithoutWaitingOutTheTimeout()
+    {
+        RecordingMxAccessServer server = new();
+        MxAccessValueCache valueCache = new();
+        using MxAccessSession session = MxAccessSession.CreateForTesting(server, new NoopEventSink(), valueCache: valueCache);
+        int serverHandle = session.Register("client");
+        string[] tags = { "Tag.1", "Tag.2", "Tag.3" };
+
+        Action pumpStep = () =>
+        {
+            foreach (string tag in tags)
+            {
+                if (server.TryGetAdvisedHandle(tag, out int handle) && !valueCache.TryGet(serverHandle, handle, out _))
+                {
+                    valueCache.Set(serverHandle, handle, BuildDataChange(serverHandle, handle, handle));
+                }
+            }
+        };
+
+        Stopwatch stopwatch = Stopwatch.StartNew();
+        IReadOnlyList<BulkReadResult> results = session.ReadBulk(serverHandle, tags, TimeSpan.FromSeconds(10), pumpStep);
+        stopwatch.Stop();
+
+        Assert.True(stopwatch.Elapsed < TimeSpan.FromSeconds(5), $"ReadBulk took {stopwatch.Elapsed.TotalMilliseconds:F0} ms.");
+        Assert.All(results, result => Assert.True(result.WasSuccessful));
+        for (int index = 0; index < tags.Length; index++)
+        {
+            Assert.Equal(results[index].ItemHandle, results[index].Value.Int32Value);
+        }
+    }
+
+    /// <summary>
+    ///     Ghost entry: an OnDataChange/OnWriteComplete already queued when
+    ///     RemoveItem ran is dispatched after the eviction and recreates the
+    ///     entries. When MXAccess later reuses that item id, AddItem must drop
+    ///     them so ReadBulk's cached path cannot serve the stale value.
+    /// </summary>
+    [Fact]
+    public void AddItem_ReusingAnEvictedHandleId_DropsGhostCacheEntries()
+    {
+        RecordingMxAccessServer server = new();
+        CompletionCacheSink sink = new();
+        MxAccessValueCache valueCache = new();
+        using MxAccessSession session = MxAccessSession.CreateForTesting(server, sink, valueCache: valueCache);
+        int serverHandle = session.Register("client");
+
+        int first = session.AddItem(serverHandle, "Tag.Old");
+        session.Advise(serverHandle, first);
+        session.RemoveItem(serverHandle, first);
+
+        // Late dispatch of callbacks queued before the RemoveItem.
+        valueCache.Set(serverHandle, first, BuildDataChange(serverHandle, first, 999));
+        RecordCompletion(sink.WriteCompletionCache, serverHandle, first);
+
+        server.NextItemHandleOverride = first;
+        int reused = session.AddItem(serverHandle, "Tag.New");
+        Assert.Equal(first, reused);
+
+        Assert.False(valueCache.TryGet(serverHandle, reused, out _));
+        Assert.Equal(0UL, sink.WriteCompletionCache.CurrentVersion(serverHandle, reused));
+
+        // Once advised, ReadBulk on the new tag does not report the ghost as cached.
+        session.Advise(serverHandle, reused);
+        BulkReadResult result = Assert.Single(session.ReadBulk(serverHandle, new[] { "Tag.New" }, TimeSpan.FromMilliseconds(30), () => { }));
+        Assert.False(result.WasCached);
+        Assert.False(result.WasSuccessful);
+    }
+
+    /// <summary>
+    ///     A handle MXAccess returns while the registry still tracks it is a
+    ///     live item, so its cached value must survive the AddItem.
+    /// </summary>
+    [Fact]
+    public void AddItem_ReturningAStillRegisteredHandle_KeepsItsCachedValue()
+    {
+        RecordingMxAccessServer server = new() { ReuseHandleForSameTag = true };
+        MxAccessValueCache valueCache = new();
+        using MxAccessSession session = MxAccessSession.CreateForTesting(server, new NoopEventSink(), valueCache: valueCache);
+        int serverHandle = session.Register("client");
+        int handle = session.AddItem(serverHandle, "Tag.Live");
+        valueCache.Set(serverHandle, handle, BuildDataChange(serverHandle, handle, 5));
+
+        Assert.Equal(handle, session.AddItem(serverHandle, "Tag.Live"));
+
+        Assert.True(valueCache.TryGet(serverHandle, handle, out _));
+    }
+
+    /// <summary>
+    ///     Duplicate addresses that MXAccess resolves to one handle: the handle
+    ///     is advised, unadvised, and removed exactly once, and every duplicate
+    ///     receives the shared outcome.
+    /// </summary>
+    [Fact]
+    public void ReadBulk_DuplicateTagsResolvingToOneHandle_AdviseAndTearDownOnceAndFanOutResult()
+    {
+        RecordingMxAccessServer server = new() { ReuseHandleForSameTag = true };
+        MxAccessValueCache valueCache = new();
+        using MxAccessSession session = MxAccessSession.CreateForTesting(server, new NoopEventSink(), valueCache: valueCache);
+        int serverHandle = session.Register("client");
+
+        Action pumpStep = () =>
+        {
+            if (server.TryGetAdvisedHandle("Tag.Dup", out int handle) && !valueCache.TryGet(serverHandle, handle, out _))
+            {
+                valueCache.Set(serverHandle, handle, BuildDataChange(serverHandle, handle, 77));
+            }
+        };
+
+        IReadOnlyList<BulkReadResult> results = session.ReadBulk(
+            serverHandle,
+            new[] { "Tag.Dup", "Tag.Other", "Tag.Dup" },
+            TimeSpan.FromMilliseconds(100),
+            pumpStep);
+
+        int dupHandle = server.HandleForTag("Tag.Dup");
+        string advise = "Advise:" + serverHandle + ":" + dupHandle;
+        string unAdvise = "UnAdvise:" + serverHandle + ":" + dupHandle;
+        string remove = "RemoveItem:" + serverHandle + ":" + dupHandle;
+        Assert.True(results[0].WasSuccessful);
+        Assert.True(results[2].WasSuccessful);
+        Assert.Equal(dupHandle, results[0].ItemHandle);
+        Assert.Equal(dupHandle, results[2].ItemHandle);
+        Assert.Equal(77, results[2].Value.Int32Value);
+        Assert.Equal("Tag.Dup", results[2].TagAddress);
+        Assert.False(results[1].WasSuccessful);
+        Assert.Equal("Tag.Other", results[1].TagAddress);
+
+        Assert.Equal(1, server.Operations.Count(op => op == advise));
+        Assert.Equal(1, server.Operations.Count(op => op == unAdvise));
+        Assert.Equal(1, server.Operations.Count(op => op == remove));
+        Assert.Empty(session.HandleRegistry.ItemHandles);
+    }
+
+    /// <summary>
+    ///     Large batches pump during the add/advise and teardown phases too,
+    ///     not only inside the wait, so the STA heartbeat stays fresh.
+    /// </summary>
+    [Fact]
+    public void ReadBulk_LargeBatch_PumpsDuringAddAdviseAndTeardownPhases()
+    {
+        RecordingMxAccessServer server = new();
+        using MxAccessSession session = MxAccessSession.CreateForTesting(server, new NoopEventSink());
+        int serverHandle = session.Register("client");
+        string[] tags = Enumerable.Range(0, 3 * MxAccessSession.ReadBulkPumpInterval).Select(i => "Tag." + i).ToArray();
+        List<int> pumpAtOperation = new();
+
+        session.ReadBulk(serverHandle, tags, TimeSpan.FromMilliseconds(20), () => pumpAtOperation.Add(server.Operations.Count));
+
+        int lastAdvise = server.Operations.FindLastIndex(op => op.StartsWith("Advise:", StringComparison.Ordinal));
+        int firstUnAdvise = server.Operations.FindIndex(op => op.StartsWith("UnAdvise:", StringComparison.Ordinal));
+        Assert.Contains(pumpAtOperation, at => at < lastAdvise);
+        Assert.Contains(pumpAtOperation, at => at > firstUnAdvise);
+    }
+
+    private static void RecordCompletion(MxAccessWriteCompletionCache cache, int serverHandle, int itemHandle)
+    {
+        RepeatedField<MxStatusProxy> statuses = new() { new MxStatusProxy { Category = MxStatusCategory.Ok } };
+        cache.Record(serverHandle, itemHandle, statuses);
+    }
+
+    private static MxEvent BuildDataChange(int serverHandle, int itemHandle, int value)
+    {
+        MxEvent mxEvent = new()
+        {
+            Family = MxEventFamily.OnDataChange,
+            ServerHandle = serverHandle,
+            ItemHandle = itemHandle,
+            Quality = 192,
+            SourceTimestamp = Timestamp.FromDateTime(new DateTime(2026, 10, 22, 0, 0, 0, DateTimeKind.Utc)),
+            Value = new MxValue
+            {
+                DataType = MxDataType.Integer,
+                VariantType = "VT_I4",
+                Int32Value = value,
+            },
+            OnDataChange = new OnDataChangeEvent(),
+        };
+        mxEvent.Statuses.Add(new MxStatusProxy { Category = MxStatusCategory.Ok });
+        return mxEvent;
+    }
+
+    /// <summary>Test sink that owns a real completion cache (Attach/Detach are no-ops).</summary>
+    private sealed class CompletionCacheSink : IMxAccessEventSink, IWriteCompletionCacheProvider
+    {
+        /// <inheritdoc />
+        public MxAccessWriteCompletionCache WriteCompletionCache { get; } = new();
+
+        /// <inheritdoc />
+        public void Attach(object mxAccessComObject, string sessionId)
+        {
+        }
+
+        /// <inheritdoc />
+        public void Detach()
+        {
+        }
+    }
+
+    /// <summary>
+    ///     Fake server that hands out increasing handles, records every
+    ///     lifecycle call, and can fail AddItem or Advise for named tags.
+    /// </summary>
+    private sealed class RecordingMxAccessServer : IMxAccessServer
+    {
+        private readonly NoopMxAccessServer noop = new();
+        private readonly Dictionary<string, int> handleByTag = new(StringComparer.Ordinal);
+        private readonly Dictionary<int, string> tagByHandle = new();
+        private readonly HashSet<int> advised = new();
+        private int nextServerHandle = 10;
+        private int nextItemHandle = 100;
+
+        public List<string> Operations { get; } = new();
+
+        public HashSet<string> FailAddItemFor { get; } = new(StringComparer.Ordinal);
+
+        public HashSet<string> FailAdviseFor { get; } = new(StringComparer.Ordinal);
+
+        /// <summary>When set, the next AddItem returns this handle (simulates MXAccess reusing an id).</summary>
+        public int? NextItemHandleOverride { get; set; }
+
+        /// <summary>When true, AddItem of an already-added address returns the existing handle.</summary>
+        public bool ReuseHandleForSameTag { get; set; }
+
+        public int HandleForTag(string tag) => handleByTag[tag];
+
+        public bool TryGetAdvisedHandle(string tag, out int handle)
+        {
+            return handleByTag.TryGetValue(tag, out handle) && advised.Contains(handle);
+        }
+
+        public int Register(string clientName)
+        {
+            Operations.Add($"Register:{clientName}");
+            return nextServerHandle++;
+        }
+
+        public void Unregister(int serverHandle) => Operations.Add($"Unregister:{serverHandle}");
+
+        public int AddItem(int serverHandle, string itemDefinition)
+        {
+            Operations.Add($"AddItem:{serverHandle}:{itemDefinition}");
+            if (FailAddItemFor.Contains(itemDefinition))
+            {
+                throw new InvalidOperationException($"AddItem failed for {itemDefinition}");
+            }
+
+            int handle;
+            if (NextItemHandleOverride is int forced)
+            {
+                NextItemHandleOverride = null;
+                handle = forced;
+            }
+            else if (ReuseHandleForSameTag && handleByTag.TryGetValue(itemDefinition, out int existing))
+            {
+                handle = existing;
+            }
+            else
+            {
+                handle = nextItemHandle++;
+            }
+
+            handleByTag[itemDefinition] = handle;
+            tagByHandle[handle] = itemDefinition;
+            return handle;
+        }
+
+        public int AddItem2(int serverHandle, string itemDefinition, string itemContext) => AddItem(serverHandle, itemDefinition);
+
+        public void RemoveItem(int serverHandle, int itemHandle) => Operations.Add($"RemoveItem:{serverHandle}:{itemHandle}");
+
+        public void Advise(int serverHandle, int itemHandle)
+        {
+            Operations.Add($"Advise:{serverHandle}:{itemHandle}");
+            if (tagByHandle.TryGetValue(itemHandle, out string? tag) && FailAdviseFor.Contains(tag))
+            {
+                throw new InvalidOperationException($"Advise failed for {tag}");
+            }
+
+            advised.Add(itemHandle);
+        }
+
+        public void UnAdvise(int serverHandle, int itemHandle)
+        {
+            Operations.Add($"UnAdvise:{serverHandle}:{itemHandle}");
+            advised.Remove(itemHandle);
+        }
+
+        public void AdviseSupervisory(int serverHandle, int itemHandle) => Advise(serverHandle, itemHandle);
+
+        public int AddBufferedItem(int serverHandle, string itemDefinition, string itemContext) => AddItem(serverHandle, itemDefinition);
+
+        public void SetBufferedUpdateInterval(int serverHandle, int updateIntervalMilliseconds) =>
+            noop.SetBufferedUpdateInterval(serverHandle, updateIntervalMilliseconds);
+
+        public object Suspend(int serverHandle, int itemHandle) => noop.Suspend(serverHandle, itemHandle);
+
+        public object Activate(int serverHandle, int itemHandle) => noop.Activate(serverHandle, itemHandle);
+
+        public void Write(int serverHandle, int itemHandle, object? value, int userId) =>
+            Operations.Add($"Write:{serverHandle}:{itemHandle}");
+
+        public void Write2(int serverHandle, int itemHandle, object? value, object? timestampValue, int userId) =>
+            Operations.Add($"Write2:{serverHandle}:{itemHandle}");
+
+        public void WriteSecured(int serverHandle, int itemHandle, int currentUserId, int verifierUserId, object? value) =>
+            Operations.Add($"WriteSecured:{serverHandle}:{itemHandle}");
+
+        public void WriteSecured2(int serverHandle, int itemHandle, int currentUserId, int verifierUserId, object? value, object? timestampValue) =>
+            Operations.Add($"WriteSecured2:{serverHandle}:{itemHandle}");
+
+        public int AuthenticateUser(int serverHandle, string verifyUser, string verifyUserPassword) => 0;
+
+        public int ArchestrAUserToId(int serverHandle, string userIdGuid) => 0;
+    }
+}

src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/MxAccessStaSessionTests.cs

@@ -1,4 +1,5 @@
 using System;
+using System.Collections.Concurrent;
 using System.Collections.Generic;
 using System.Runtime.InteropServices;
 using System.Threading;
@@ -573,6 +574,181 @@ public sealed class MxAccessStaSessionTests
             "Expected StaCallInProgress to clear once the alarm poll returned.");
     }
 
+    /// <summary>
+    /// A transient PollOnce failure must not fault the whole session: with a
+    /// failure threshold of 3, a poll pattern that never fails three times in
+    /// a row (fail, fail, succeed, ...) keeps the session healthy, and each
+    /// transient failure is logged with type/HRESULT only — never the
+    /// exception message.
+    /// </summary>
+    /// <returns>A task that represents the asynchronous operation.</returns>
+    [Fact]
+    public async Task RunAlarmPollLoop_TransientFailuresBelowThreshold_DoNotFaultSession()
+    {
+        FakeAlarmCommandHandler handler = new()
+        {
+            PollExceptionFactory = call => call % 3 == 0
+                ? null
+                : new COMException("secret-tag-name GetXmlCurrentAlarms2 failed.", unchecked((int)0x80004005)),
+        };
+        ConcurrentQueue<string> warnings = new();
+        using StaRuntime runtime = CreateRuntime();
+        MxAccessEventQueue eventQueue = new();
+        using MxAccessStaSession session = new(
+            runtime,
+            new FakeMxAccessComObjectFactory(),
+            new FakeMxAccessEventSink(),
+            eventQueue,
+            (_eq, _affinity, _comFactory) => handler);
+        session.AlarmPollInterval = TimeSpan.FromMilliseconds(10);
+        session.AlarmPollFailureThreshold = 3;
+        session.AlarmPollWarningSink = warnings.Enqueue;
+
+        await session.StartAsync("session-1", workerProcessId: 1);
+
+        DateTime deadline = DateTime.UtcNow.AddSeconds(10);
+        while (handler.PollCount < 12 && !eventQueue.IsFaulted && DateTime.UtcNow < deadline)
+        {
+            await Task.Delay(10);
+        }
+
+        Assert.False(eventQueue.IsFaulted, "Transient alarm-poll failures below the threshold must not fault the session.");
+        Assert.True(handler.PollCount >= 12, $"Expected the poll loop to keep polling; saw {handler.PollCount} polls.");
+        Assert.NotEmpty(warnings);
+        Assert.All(warnings, line =>
+        {
+            Assert.Contains("event=AlarmPollFailed", line, StringComparison.Ordinal);
+            Assert.Contains("hresult=0x80004005", line, StringComparison.Ordinal);
+            Assert.DoesNotContain("secret-tag-name", line, StringComparison.Ordinal);
+        });
+    }
+
+    /// <summary>
+    /// A run of failures reaching the threshold with no success in between
+    /// faults the session exactly once and stops the loop: two warnings, then
+    /// the fault on the third consecutive failure, then no further polls.
+    /// </summary>
+    /// <returns>A task that represents the asynchronous operation.</returns>
+    [Fact]
+    public async Task RunAlarmPollLoop_ConsecutiveFailuresReachThreshold_FaultsAndStops()
+    {
+        FakeAlarmCommandHandler handler = new()
+        {
+            PollException = new COMException("GetXmlCurrentAlarms2 failed.", unchecked((int)0x80004005)),
+        };
+        ConcurrentQueue<string> warnings = new();
+        using StaRuntime runtime = CreateRuntime();
+        MxAccessEventQueue eventQueue = new();
+        using MxAccessStaSession session = new(
+            runtime,
+            new FakeMxAccessComObjectFactory(),
+            new FakeMxAccessEventSink(),
+            eventQueue,
+            (_eq, _affinity, _comFactory) => handler);
+        session.AlarmPollInterval = TimeSpan.FromMilliseconds(10);
+        session.AlarmPollFailureThreshold = 3;
+        session.AlarmPollWarningSink = warnings.Enqueue;
+
+        await session.StartAsync("session-1", workerProcessId: 1);
+
+        DateTime deadline = DateTime.UtcNow.AddSeconds(10);
+        while (!eventQueue.IsFaulted && DateTime.UtcNow < deadline)
+        {
+            await Task.Delay(10);
+        }
+
+        Assert.True(eventQueue.IsFaulted);
+        await Task.Delay(200);
+        Assert.Equal(3, handler.PollCount);
+        Assert.Equal(2, warnings.Count);
+        WorkerFault? fault = session.DrainFault();
+        Assert.NotNull(fault);
+        Assert.Equal(WorkerFaultCategory.MxaccessEventConversionFailed, fault!.Category);
+    }
+
+    /// <summary>
+    /// WRK-02: when the STA loop dies after the session started, the death is
+    /// recorded as a StaHung fault on the event queue — the path the IPC drain
+    /// loop turns into a WorkerFault frame and a terminated pipe session.
+    /// </summary>
+    /// <returns>A task that represents the asynchronous operation.</returns>
+    [Fact]
+    public async Task StaLoopDeath_AfterStart_RecordsStaHungFaultOnEventQueue()
+    {
+        ControllableMessagePump pump = new();
+        using StaRuntime runtime = new(new NoopComApartmentInitializer(), pump, TimeSpan.FromMilliseconds(25));
+        MxAccessEventQueue eventQueue = new();
+        using MxAccessStaSession session = new(
+            runtime,
+            new FakeMxAccessComObjectFactory(),
+            new FakeMxAccessEventSink(),
+            eventQueue);
+
+        await session.StartAsync("session-1", workerProcessId: 1);
+        Assert.False(eventQueue.IsFaulted);
+
+        pump.ThrowOnNextPump(new InvalidOperationException("secret-tag-name pump failed."));
+
+        DateTime deadline = DateTime.UtcNow.AddSeconds(5);
+        while (!eventQueue.IsFaulted && DateTime.UtcNow < deadline)
+        {
+            await Task.Delay(10);
+        }
+
+        Assert.True(eventQueue.IsFaulted, "Expected the STA loop death to fault the event queue.");
+        WorkerFault? fault = session.DrainFault();
+        Assert.NotNull(fault);
+        Assert.Equal(WorkerFaultCategory.StaHung, fault!.Category);
+        Assert.Equal(typeof(InvalidOperationException).FullName, fault.ExceptionType);
+        Assert.Contains("STA loop terminated", fault.DiagnosticMessage, StringComparison.Ordinal);
+        Assert.Contains("exceptionType=System.InvalidOperationException", fault.DiagnosticMessage, StringComparison.Ordinal);
+        Assert.Contains("hresult=0x", fault.DiagnosticMessage, StringComparison.Ordinal);
+        Assert.DoesNotContain("secret-tag-name", fault.DiagnosticMessage, StringComparison.Ordinal);
+        Assert.DoesNotContain("secret-tag-name", fault.ProtocolStatus.Message, StringComparison.Ordinal);
+        Assert.Equal(ProtocolStatusCode.WorkerUnavailable, fault.ProtocolStatus.Code);
+    }
+
+    /// <summary>
+    /// After the STA loop dies, the alarm poll's fast-failed InvokeAsync must
+    /// not record a second, mislabelled alarm-poll fault: once the StaHung
+    /// fault is drained, nothing else is recorded.
+    /// </summary>
+    /// <returns>A task that represents the asynchronous operation.</returns>
+    [Fact]
+    public async Task RunAlarmPollLoop_AfterStaLoopFault_DoesNotRecordAlarmPollFault()
+    {
+        ControllableMessagePump pump = new();
+        FakeAlarmCommandHandler handler = new();
+        using StaRuntime runtime = new(new NoopComApartmentInitializer(), pump, TimeSpan.FromMilliseconds(25));
+        MxAccessEventQueue eventQueue = new();
+        using MxAccessStaSession session = new(
+            runtime,
+            new FakeMxAccessComObjectFactory(),
+            new FakeMxAccessEventSink(),
+            eventQueue,
+            (_eq, _affinity, _comFactory) => handler);
+        session.AlarmPollInterval = TimeSpan.FromMilliseconds(10);
+        session.AlarmPollFailureThreshold = 1;
+        session.AlarmPollWarningSink = _ => { };
+
+        await session.StartAsync("session-1", workerProcessId: 1);
+        pump.ThrowOnNextPump(new InvalidOperationException("pump failed"));
+
+        DateTime deadline = DateTime.UtcNow.AddSeconds(5);
+        while (!eventQueue.IsFaulted && DateTime.UtcNow < deadline)
+        {
+            await Task.Delay(10);
+        }
+
+        WorkerFault? first = session.DrainFault();
+        Assert.NotNull(first);
+        Assert.Equal(WorkerFaultCategory.StaHung, first!.Category);
+
+        // Give the poll loop several ticks to (wrongly) record its own fault.
+        await Task.Delay(300);
+        Assert.Null(session.DrainFault());
+    }
+
     /// <summary>
     /// The STA-affinity guard throws when an
     /// IMxAccessAlarmConsumer call is attempted off the thread that created
@@ -621,6 +797,9 @@ public sealed class MxAccessStaSessionTests
         /// <summary>Exception thrown by PollOnce; null to succeed.</summary>
         public Exception? PollException { get; set; }
 
+        /// <summary>Per-call exception factory (1-based call number); null result succeeds.</summary>
+        public Func<int, Exception?>? PollExceptionFactory { get; set; }
+
         /// <summary>When set, <see cref="PollOnce"/> blocks until <see cref="ReleasePoll"/> is called.</summary>
         public bool BlockPoll { get; set; }
 
@@ -699,6 +878,22 @@ public sealed class MxAccessStaSessionTests
             {
                 throw PollException;
             }
+
+            Func<int, Exception?>? factory = PollExceptionFactory;
+            if (factory is not null)
+            {
+                int callNumber;
+                lock (gate)
+                {
+                    callNumber = pollCount;
+                }
+
+                Exception? exception = factory(callNumber);
+                if (exception is not null)
+                {
+                    throw exception;
+                }
+            }
         }
 
         /// <inheritdoc />

src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/MxAccessWriteCompletionCacheTests.cs

@@ -183,6 +183,39 @@ public sealed class MxAccessWriteCompletionCacheTests
         Assert.Equal(77, Assert.Single(statuses).Detail);
     }
 
+    /// <summary>Verifies that Remove forgets only the given handle pair.</summary>
+    [Fact]
+    public void Remove_ForgetsOnlyThatHandlePair()
+    {
+        MxAccessWriteCompletionCache cache = new();
+        cache.Record(7, 21, BuildStatuses(detail: 1));
+        cache.Record(7, 22, BuildStatuses(detail: 2));
+
+        cache.Remove(7, 21);
+        cache.Remove(7, 99); // unknown pair is a no-op
+
+        Assert.Equal(1, cache.Count);
+        Assert.Equal(0UL, cache.CurrentVersion(7, 21));
+        Assert.Equal(1UL, cache.CurrentVersion(7, 22));
+    }
+
+    /// <summary>Verifies that RemoveServer forgets every pair of that server and no other.</summary>
+    [Fact]
+    public void RemoveServer_ForgetsOnlyThatServersPairs()
+    {
+        MxAccessWriteCompletionCache cache = new();
+        cache.Record(7, 21, BuildStatuses(detail: 1));
+        cache.Record(7, -5, BuildStatuses(detail: 2));
+        cache.Record(8, 21, BuildStatuses(detail: 3));
+        cache.Record(-7, 21, BuildStatuses(detail: 4));
+
+        cache.RemoveServer(7);
+
+        Assert.Equal(2, cache.Count);
+        Assert.Equal(1UL, cache.CurrentVersion(8, 21));
+        Assert.Equal(1UL, cache.CurrentVersion(-7, 21));
+    }
+
     private static RepeatedField<MxStatusProxy> BuildStatuses(int detail)
     {
         return new RepeatedField<MxStatusProxy>

src/ZB.MOM.WW.MxGateway.Worker.Tests/Probes/ItemHandleReuseProbeTests.cs

@@ -0,0 +1,185 @@
+// ItemHandleReuseProbeTests.cs
+//
+// Probe for issue #166: does MXAccess hand out a new item handle number when the
+// same item is removed and re-added on one LMXProxyServer registration? The
+// AlarmAckWriter's Unconfirmed mitigation (drop the item, re-add it, wait on the
+// fresh handle) relies on the number changing, because OnWriteComplete carries
+// only the (server, item) handle pair.
+//
+// The probe only adds, advises and removes items. It never writes, so it does
+// not acknowledge the alarm or create Historian rows.
+//
+// HOW TO RUN (dev rig with MXAccess installed and the DEV Galaxy running):
+//   1. Remove (or set to null) the Skip parameter on the [Fact] below.
+//   2. dotnet test src/ZB.MOM.WW.MxGateway.Worker.Tests/ZB.MOM.WW.MxGateway.Worker.Tests.csproj
+//        -p:Platform=x86 --filter FullyQualifiedName~ItemHandleReuseProbeTests
+//        --logger "console;verbosity=detailed"
+//   3. Read the logged handles. Results are recorded in docs/AlarmProbeFindings.md.
+//
+// RESULT (windev, 2026-10-20): within one registration item handles increase
+// monotonically and are never reused (1..15 across add/remove/re-add of the same
+// item, with and without Advise, interleaved with another item, and while held
+// twice). Item numbering restarts at 1 after Unregister + Register, and a fresh
+// COM object is handed server handle 1 again.
+
+using System;
+using System.Collections.Generic;
+using System.Runtime.InteropServices;
+using System.Threading;
+using ArchestrA.MxAccess;
+using ZB.MOM.WW.MxGateway.Worker.MxAccess;
+using Xunit.Abstractions;
+
+namespace ZB.MOM.WW.MxGateway.Worker.Tests.Probes;
+
+/// <summary>
+///     Live probe that logs the item handles MXAccess returns when the same
+///     <c>AckMsg</c> item is removed and re-added, to decide whether a late
+///     <c>OnWriteComplete</c> for a dropped handle could match a re-added one.
+/// </summary>
+public sealed class ItemHandleReuseProbeTests
+{
+    private const string AckItem = "TestMachine_001.TestAlarm001.AckMsg";
+    private const string OtherItem = "TestMachine_001.TestAlarm001.Acked";
+    private const int Rounds = 5;
+
+    private readonly ITestOutputHelper output;
+    private readonly List<string> log = new List<string>();
+
+    // Item handles returned on the first registration, in the order they were handed out.
+    private readonly List<int> firstRegistrationHandles = new List<int>();
+
+    /// <summary>Initializes a new instance of the <see cref="ItemHandleReuseProbeTests"/> class.</summary>
+    /// <param name="output">Test output helper for logging.</param>
+    public ItemHandleReuseProbeTests(ITestOutputHelper output)
+    {
+        this.output = output;
+    }
+
+    /// <summary>Adds, removes and re-adds the AckMsg item on a fresh registration and logs every handle.</summary>
+    [Fact(Skip = "Live dev-rig probe (issue #166) — flip Skip=null on a host with MXAccess and the DEV Galaxy running.")]
+    public void RemoveThenReAdd_LogsItemHandles()
+    {
+        Exception? threadException = null;
+        Thread thread = new Thread(() =>
+        {
+            try { RunProbe(); }
+            catch (Exception ex) { threadException = ex; }
+        });
+        thread.SetApartmentState(ApartmentState.STA);
+        thread.Start();
+        thread.Join();
+
+        foreach (string line in log)
+        {
+            output.WriteLine(line);
+        }
+
+        if (threadException is not null)
+        {
+            throw new InvalidOperationException("Probe failed: " + threadException.Message, threadException);
+        }
+
+        // The ack writer's drop-and-re-add mitigation depends on this: within one
+        // registration a handle number is never handed out twice.
+        Assert.Equal((2 * Rounds) + 5, firstRegistrationHandles.Count);
+        for (int i = 1; i < firstRegistrationHandles.Count; i++)
+        {
+            Assert.True(
+                firstRegistrationHandles[i] > firstRegistrationHandles[i - 1],
+                $"Item handle {firstRegistrationHandles[i]} did not increase past {firstRegistrationHandles[i - 1]}; MXAccess reused or reordered item handles within one registration.");
+        }
+    }
+
+    private void RunProbe()
+    {
+        LMXProxyServerClass com = new LMXProxyServerClass();
+        try
+        {
+            IMxAccessServer mx = new MxAccessComServer(com);
+            int server = mx.Register("MxGatewayProbe166");
+            Log($"Register -> server handle {server}");
+
+            // A: the AlarmAckWriter drop path (Advise, then UnAdvise + RemoveItem).
+            for (int i = 0; i < Rounds; i++)
+            {
+                int h = Held(mx.AddItem(server, AckItem));
+                mx.Advise(server, h);
+                Log($"A{i}: AddItem+Advise {AckItem} -> {h}");
+                mx.UnAdvise(server, h);
+                mx.RemoveItem(server, h);
+                Log($"A{i}: UnAdvise+RemoveItem {h}");
+            }
+
+            // B: RemoveItem without Advise.
+            for (int i = 0; i < Rounds; i++)
+            {
+                int h = Held(mx.AddItem(server, AckItem));
+                Log($"B{i}: AddItem {AckItem} -> {h}");
+                mx.RemoveItem(server, h);
+                Log($"B{i}: RemoveItem {h}");
+            }
+
+            // C: interleave another item to see whether a freed number is handed to any item.
+            int c1 = Held(mx.AddItem(server, AckItem));
+            Log($"C: AddItem {AckItem} -> {c1}");
+            mx.RemoveItem(server, c1);
+            Log($"C: RemoveItem {c1}");
+            int c2 = Held(mx.AddItem(server, OtherItem));
+            Log($"C: AddItem {OtherItem} -> {c2}");
+            int c3 = Held(mx.AddItem(server, AckItem));
+            Log($"C: AddItem {AckItem} -> {c3}");
+            mx.RemoveItem(server, c2);
+            mx.RemoveItem(server, c3);
+            Log($"C: RemoveItem {c2}, {c3}");
+
+            // D: the same item added twice while the first is still held.
+            int d1 = Held(mx.AddItem(server, AckItem));
+            int d2 = Held(mx.AddItem(server, AckItem));
+            Log($"D: AddItem {AckItem} twice while held -> {d1}, {d2}");
+            mx.RemoveItem(server, d1);
+            mx.RemoveItem(server, d2);
+
+            mx.Unregister(server);
+            Log($"Unregister {server}");
+
+            // E: the TearDownServer path: re-register on the same COM object and add again.
+            int server2 = mx.Register("MxGatewayProbe166");
+            int e1 = mx.AddItem(server2, AckItem);
+            Log($"E: re-Register -> server handle {server2}; AddItem {AckItem} -> {e1}");
+            mx.RemoveItem(server2, e1);
+            mx.Unregister(server2);
+        }
+        finally
+        {
+            Marshal.FinalReleaseComObject(com);
+        }
+
+        // F: a fresh COM object, as AlarmAckWriter creates after a teardown.
+        LMXProxyServerClass com2 = new LMXProxyServerClass();
+        try
+        {
+            IMxAccessServer mx2 = new MxAccessComServer(com2);
+            int server3 = mx2.Register("MxGatewayProbe166");
+            int f1 = mx2.AddItem(server3, AckItem);
+            Log($"F: new COM object Register -> server handle {server3}; AddItem {AckItem} -> {f1}");
+            mx2.RemoveItem(server3, f1);
+            mx2.Unregister(server3);
+        }
+        finally
+        {
+            Marshal.FinalReleaseComObject(com2);
+        }
+    }
+
+    private int Held(int itemHandle)
+    {
+        firstRegistrationHandles.Add(itemHandle);
+        return itemHandle;
+    }
+
+    private void Log(string line)
+    {
+        log.Add(DateTime.Now.ToString("HH:mm:ss.fff") + " " + line);
+    }
+}

src/ZB.MOM.WW.MxGateway.Worker.Tests/Sta/StaRuntimeTests.cs

@@ -2,6 +2,7 @@ using System;
 using System.Threading;
 using System.Threading.Tasks;
 using ZB.MOM.WW.MxGateway.Worker.Sta;
+using ZB.MOM.WW.MxGateway.Worker.Tests.TestSupport;
 
 namespace ZB.MOM.WW.MxGateway.Worker.Tests.Sta;
 
@@ -155,6 +156,145 @@ public sealed class StaRuntimeTests
         Assert.Contains("shutting down", exception.Message);
     }
 
+    /// <summary>
+    ///     A backlog of queued work items must not starve inbound COM
+    ///     callbacks: the loop pumps Windows messages between items rather
+    ///     than only after draining the whole queue. A gate item holds the STA
+    ///     while two more items queue behind it; once released, the second
+    ///     queued item must observe a pump that happened after the first.
+    /// </summary>
+    /// <returns>A task that represents the asynchronous operation.</returns>
+    [Fact]
+    public async Task ProcessQueuedCommands_PumpsMessagesBetweenQueuedWorkItems()
+    {
+        ControllableMessagePump pump = new();
+        using StaRuntime runtime = new(new RecordingComApartmentInitializer(), pump, TimeSpan.FromSeconds(30));
+        runtime.Start();
+
+        using ManualResetEventSlim gateEntered = new(false);
+        using ManualResetEventSlim releaseGate = new(false);
+        Task gate = runtime.InvokeAsync(() =>
+        {
+            gateEntered.Set();
+            releaseGate.Wait(TimeSpan.FromSeconds(10));
+        });
+        Assert.True(gateEntered.Wait(TimeSpan.FromSeconds(5)));
+
+        Task<int> first = runtime.InvokeAsync(() => pump.PumpCount);
+        Task<int> second = runtime.InvokeAsync(() => pump.PumpCount);
+        releaseGate.Set();
+
+        await gate;
+        int pumpsSeenByFirst = await first;
+        int pumpsSeenBySecond = await second;
+
+        Assert.True(
+            pumpsSeenBySecond > pumpsSeenByFirst,
+            $"Expected a pump between queued work items (first saw {pumpsSeenByFirst}, second saw {pumpsSeenBySecond}).");
+    }
+
+    /// <summary>
+    ///     WRK-02: an exception that kills the STA loop after a successful
+    ///     <see cref="StaRuntime.Start"/> must be surfaced — recorded on
+    ///     <see cref="StaRuntime.LoopFault"/>, handed to the registered fault
+    ///     handler, and turned into fast failures for every later
+    ///     <see cref="StaRuntime.InvokeAsync{T}(Func{T}, CancellationToken)"/> —
+    ///     rather than parked in a startup field nothing reads.
+    /// </summary>
+    /// <returns>A task that represents the asynchronous operation.</returns>
+    [Fact]
+    public async Task LoopException_AfterStart_IsReportedToHandlerAndFailsLaterCommands()
+    {
+        ControllableMessagePump pump = new();
+        RecordingComApartmentInitializer initializer = new();
+        using StaRuntime runtime = new(initializer, pump, TimeSpan.FromMilliseconds(25));
+        runtime.Start();
+
+        TaskCompletionSource<Exception> reported = new(TaskCreationOptions.RunContinuationsAsynchronously);
+        runtime.RegisterLoopFaultHandler(exception => reported.TrySetResult(exception));
+        Assert.Null(runtime.LoopFault);
+
+        InvalidOperationException loopException = new("pump failed");
+        pump.ThrowOnNextPump(loopException);
+
+        Task completed = await Task.WhenAny(reported.Task, Task.Delay(TimeSpan.FromSeconds(5)));
+        Assert.Same(reported.Task, completed);
+        Assert.Same(loopException, await reported.Task);
+        Assert.Same(loopException, runtime.LoopFault);
+
+        InvalidOperationException rejected = await Assert.ThrowsAsync<InvalidOperationException>(
+            () => runtime.InvokeAsync(() => 1));
+        Assert.IsNotType<StaRuntimeShutdownException>(rejected);
+        Assert.Same(loopException, rejected.InnerException);
+
+        // The dying thread still runs its apartment cleanup.
+        Assert.True(runtime.Shutdown(TimeSpan.FromSeconds(5)));
+        Assert.False(runtime.IsRunning);
+        Assert.Equal(1, initializer.UninitializeCount);
+    }
+
+    /// <summary>
+    ///     A handler registered after the loop already died still receives the
+    ///     fault, so the window between Start and registration loses nothing.
+    /// </summary>
+    /// <returns>A task that represents the asynchronous operation.</returns>
+    [Fact]
+    public async Task RegisterLoopFaultHandler_AfterLoopDied_InvokesHandlerImmediately()
+    {
+        ControllableMessagePump pump = new();
+        using StaRuntime runtime = new(new RecordingComApartmentInitializer(), pump, TimeSpan.FromMilliseconds(25));
+        runtime.Start();
+        InvalidOperationException loopException = new("pump failed");
+        pump.ThrowOnNextPump(loopException);
+
+        DateTime deadline = DateTime.UtcNow.AddSeconds(5);
+        while (runtime.LoopFault is null && DateTime.UtcNow < deadline)
+        {
+            await Task.Delay(10);
+        }
+
+        Exception? seen = null;
+        runtime.RegisterLoopFaultHandler(exception => seen = exception);
+
+        Assert.Same(loopException, seen);
+    }
+
+    /// <summary>A requested shutdown is not a loop fault.</summary>
+    [Fact]
+    public void Shutdown_DoesNotRecordLoopFault()
+    {
+        using StaRuntime runtime = new(new RecordingComApartmentInitializer(), new ControllableMessagePump(), TimeSpan.FromMilliseconds(25));
+        runtime.Start();
+        bool invoked = false;
+        runtime.RegisterLoopFaultHandler(_ => invoked = true);
+
+        Assert.True(runtime.Shutdown(TimeSpan.FromSeconds(5)));
+
+        Assert.Null(runtime.LoopFault);
+        Assert.False(invoked);
+    }
+
+    /// <summary>
+    ///     A loop that throws while a requested shutdown is in progress is
+    ///     being torn down on purpose: the loop-fault handler must not run.
+    /// </summary>
+    [Fact]
+    public void LoopException_DuringRequestedShutdown_DoesNotInvokeHandler()
+    {
+        ControllableMessagePump pump = new();
+        using StaRuntime runtime = new(new RecordingComApartmentInitializer(), pump, TimeSpan.FromSeconds(30));
+        runtime.Start();
+        bool invoked = false;
+        runtime.RegisterLoopFaultHandler(_ => invoked = true);
+
+        // The loop is parked in the 30 s idle wait; Shutdown wakes it and the
+        // very next pump throws with shutdownRequested already set.
+        pump.ThrowOnNextPump(new InvalidOperationException("pump failed during shutdown"));
+        Assert.True(runtime.Shutdown(TimeSpan.FromSeconds(5)));
+
+        Assert.False(invoked);
+    }
+
     private static StaRuntime CreateRuntime(RecordingComApartmentInitializer initializer)
     {
         return new StaRuntime(

src/ZB.MOM.WW.MxGateway.Worker.Tests/TestSupport/ControllableMessagePump.cs

@@ -0,0 +1,47 @@
+using System;
+using System.Threading;
+using ZB.MOM.WW.MxGateway.Worker.Sta;
+
+namespace ZB.MOM.WW.MxGateway.Worker.Tests.TestSupport;
+
+/// <summary>
+/// Test <see cref="IStaMessagePump"/> that counts pump calls and can be armed
+/// to throw from the next pump, so tests can observe the STA loop's pump
+/// interleaving and drive a post-start loop death without a real Win32
+/// message queue. The idle wait blocks on the command wake event exactly
+/// like the production pump's <c>MsgWaitForMultipleObjectsEx</c> does.
+/// </summary>
+internal sealed class ControllableMessagePump : IStaMessagePump
+{
+    private int pumpCount;
+    private Exception? throwOnNextPump;
+
+    /// <summary>Gets the number of <see cref="PumpPendingMessages"/> calls so far.</summary>
+    public int PumpCount => Volatile.Read(ref pumpCount);
+
+    /// <summary>Arms the pump to throw <paramref name="exception"/> from its next pump call.</summary>
+    /// <param name="exception">The exception to throw once.</param>
+    public void ThrowOnNextPump(Exception exception)
+    {
+        Volatile.Write(ref throwOnNextPump, exception);
+    }
+
+    /// <inheritdoc />
+    public void WaitForWorkOrMessages(WaitHandle commandWakeEvent, TimeSpan timeout)
+    {
+        commandWakeEvent.WaitOne(timeout);
+    }
+
+    /// <inheritdoc />
+    public int PumpPendingMessages()
+    {
+        Interlocked.Increment(ref pumpCount);
+        Exception? exception = Interlocked.Exchange(ref throwOnNextPump, null);
+        if (exception is not null)
+        {
+            throw exception;
+        }
+
+        return 0;
+    }
+}

src/ZB.MOM.WW.MxGateway.Worker/Bootstrap/WorkerExitCode.cs

@@ -9,4 +9,13 @@ public enum WorkerExitCode
     MissingNonce = 4,
     PipeConnectionFailed = 5,
     ProtocolViolation = 6,
+
+    /// <summary>
+    ///     An exception escaped to <c>AppDomain.UnhandledException</c> on some
+    ///     thread (a background task, a COM callback thread, a timer) rather
+    ///     than through <c>WorkerApplication.Run</c>'s own catch blocks. The
+    ///     worker logs it and exits with this code instead of the CLR's crash
+    ///     exit code.
+    /// </summary>
+    UnhandledException = 7,
 }

src/ZB.MOM.WW.MxGateway.Worker/Ipc/WorkerPipeSession.cs

@@ -76,7 +76,7 @@ public sealed class WorkerPipeSession
             options,
             () => Process.GetCurrentProcess().Id,
             new WorkerPipeSessionOptions(),
-            () => new MxAccessStaSession((eq, affinity, comFactory) => new AlarmCommandHandler(eq, () => new WnWrapAlarmConsumer(), affinity, comFactory, standbyFactory: null)),
+            () => new MxAccessStaSession((eq, affinity, comFactory, pump) => new AlarmCommandHandler(eq, () => new WnWrapAlarmConsumer(), affinity, comFactory, standbyFactory: null, ackPumpStep: pump)),
             logger,
             stream)
     {
@@ -98,7 +98,7 @@ public sealed class WorkerPipeSession
             options,
             processIdProvider,
             new WorkerPipeSessionOptions(),
-            () => new MxAccessStaSession((eq, affinity, comFactory) => new AlarmCommandHandler(eq, () => new WnWrapAlarmConsumer(), affinity, comFactory, standbyFactory: null)),
+            () => new MxAccessStaSession((eq, affinity, comFactory, pump) => new AlarmCommandHandler(eq, () => new WnWrapAlarmConsumer(), affinity, comFactory, standbyFactory: null, ackPumpStep: pump)),
             logger: null)
     {
     }
@@ -1585,7 +1585,7 @@ public sealed class WorkerPipeSession
         // parameterless CompleteStartupHandshakeAsync is used without a
         // prior factory call.
         _runtimeSession ??= new MxAccessStaSession(
-            (eq, affinity, comFactory) => new AlarmCommandHandler(eq, () => new WnWrapAlarmConsumer(), affinity, comFactory, standbyFactory: null));
+            (eq, affinity, comFactory, pump) => new AlarmCommandHandler(eq, () => new WnWrapAlarmConsumer(), affinity, comFactory, standbyFactory: null, ackPumpStep: pump));
         IWorkerRuntimeSession session = _runtimeSession;
         try
         {

src/ZB.MOM.WW.MxGateway.Worker/MxAccess/AlarmAckWriter.cs

@@ -26,13 +26,30 @@ public sealed class AlarmAckWriter : IAlarmAckWriter
     /// <summary>Default MXAccess client name when none is configured.</summary>
     public const string DefaultClientName = "MxGateway";
 
+    /// <summary>
+    ///     Default cap on cached ack item handles. Each cached item holds an
+    ///     advised MXAccess item for the worker's life, so the cache is bounded
+    ///     (least-recently-used eviction) rather than growing with every
+    ///     distinct alarm ever acknowledged.
+    /// </summary>
+    public const int DefaultMaxCachedItems = 256;
+
     private readonly IMxAccessComObjectFactory? factory;
     private readonly MxAccessWriteCompletionCache completionCache;
     private readonly Action pumpStep;
     private readonly TimeSpan completionTimeout;
     private readonly string clientName;
     private readonly MxStatusProxyConverter statusConverter = new MxStatusProxyConverter();
-    private readonly Dictionary<string, int> itemHandles = new Dictionary<string, int>(StringComparer.OrdinalIgnoreCase);
+    private readonly int maxCachedItems;
+    private readonly Dictionary<string, LinkedListNode<AckItem>> itemHandles = new Dictionary<string, LinkedListNode<AckItem>>(StringComparer.OrdinalIgnoreCase);
+
+    // Most-recently-used first; the tail is the eviction candidate.
+    private readonly LinkedList<AckItem> itemLru = new LinkedList<AckItem>();
+
+    // Item handles currently added on this registration. A completion for any
+    // other handle (a dropped one whose OnWriteComplete arrives late) is ignored
+    // so it cannot recreate a completion-cache entry for a dead handle.
+    private readonly HashSet<int> heldItemHandles = new HashSet<int>();
     private object? comObject;
     private LMXProxyServerClass? comEventSource;
     private IMxAccessServer? server;
@@ -54,6 +71,7 @@ public sealed class AlarmAckWriter : IAlarmAckWriter
         this.pumpStep = pumpStep ?? throw new ArgumentNullException(nameof(pumpStep));
         this.completionTimeout = completionTimeout;
         this.clientName = string.IsNullOrWhiteSpace(clientName) ? DefaultClientName : clientName!;
+        maxCachedItems = DefaultMaxCachedItems;
         completionCache = new MxAccessWriteCompletionCache();
     }
 
@@ -63,8 +81,11 @@ public sealed class AlarmAckWriter : IAlarmAckWriter
         MxAccessWriteCompletionCache completionCache,
         Action pumpStep,
         TimeSpan completionTimeout,
-        string clientName)
+        string clientName,
+        int maxCachedItems = DefaultMaxCachedItems)
     {
+        if (maxCachedItems < 1) throw new ArgumentOutOfRangeException(nameof(maxCachedItems));
+        this.maxCachedItems = maxCachedItems;
         this.server = server ?? throw new ArgumentNullException(nameof(server));
         this.completionCache = completionCache ?? throw new ArgumentNullException(nameof(completionCache));
         this.pumpStep = pumpStep ?? throw new ArgumentNullException(nameof(pumpStep));
@@ -88,19 +109,36 @@ public sealed class AlarmAckWriter : IAlarmAckWriter
         if (string.IsNullOrWhiteSpace(itemAddress)) throw new ArgumentException("Ack item address is required.", nameof(itemAddress));
         if (ackOperator is null) throw new ArgumentNullException(nameof(ackOperator));
 
+        if (ackOperator.UserName is null) throw new ArgumentException("Operator user name is required.", nameof(ackOperator));
+        string verifyUser = ackOperator.VerifyUser;
+        string password = ackOperator.Password ?? string.Empty;
+        string commentValue = comment ?? string.Empty;
+
         IMxAccessServer mx = EnsureRegistered();
 
         int userId;
         try
         {
-            userId = mx.AuthenticateUser(serverHandle, ackOperator.VerifyUser, ackOperator.Password ?? string.Empty);
+            userId = Interop("AuthenticateUser", () => mx.AuthenticateUser(serverHandle, verifyUser, password));
         }
-        catch (COMException ex)
+        catch (AckInteropException ex)
         {
+            int authHresult = ex.HResult;
+            if (IsServerGone(authHresult))
+            {
+                // The registration is dead, not the credentials: tear down so the
+                // next ack re-registers, and report it as a write-path failure.
+                TearDownServer();
+                return AlarmAckResult.Create(
+                    AlarmAckOutcome.WriteRejected,
+                    authHresult,
+                    $"AckMsg AuthenticateUser failed (HRESULT 0x{authHresult:X8}).");
+            }
+
             return AlarmAckResult.Create(
                 AlarmAckOutcome.AuthenticationFailed,
                 ex.HResult,
-                $"Operator authentication failed for '{ackOperator.VerifyUser}' (HRESULT 0x{ex.HResult:X8}).");
+                $"Operator authentication failed for '{verifyUser}' (HRESULT 0x{ex.HResult:X8}).");
         }
 
         if (userId == 0)
@@ -108,36 +146,59 @@ public sealed class AlarmAckWriter : IAlarmAckWriter
             return AlarmAckResult.Create(
                 AlarmAckOutcome.AuthenticationFailed,
                 -1,
-                $"Operator authentication failed for '{ackOperator.VerifyUser}' (no user id returned).");
+                $"Operator authentication failed for '{verifyUser}' (no user id returned).");
         }
 
         ulong baselineVersion = 0;
         int itemHandle = 0;
-        bool cached = itemHandles.TryGetValue(itemAddress, out itemHandle);
+        LinkedListNode<AckItem>? node;
-        string step = "AddItem";
+        bool cached = itemHandles.TryGetValue(itemAddress, out node);
         try
         {
-            if (!cached)
+            if (cached && node!.Value.Dirty)
             {
-                itemHandle = mx.AddItem(serverHandle, itemAddress);
+                // The previous ack to this item ended Unconfirmed, so its
-                itemHandles[itemAddress] = itemHandle;
+                // OnWriteComplete may still arrive. Completions are keyed by
-                step = "Advise";
+                // (server, item) handle with no per-write id, so a late one would
+                // be credited to this write. Drop the item and re-add it so this
+                // write waits on a fresh handle the stale completion cannot match.
+                // That holds because MXAccess never reuses an item handle number
+                // within one registration: the live probe (ItemHandleReuseProbeTests,
+                // windev 2026-10-20) saw strictly increasing handles across repeated
+                // remove/re-add of the same item. Numbering restarts only on a new
+                // registration, and TearDownServer detaches this writer's
+                // OnWriteComplete handler and clears the server's completion entries
+                // first, so the old registration's completions cannot reach it.
+                DropItem(mx, node);
+                cached = false;
+            }
+
+            if (cached)
+            {
+                itemHandle = node!.Value.Handle;
+                itemLru.Remove(node);
+                itemLru.AddFirst(node);
+            }
+            else
+            {
+                itemHandle = Interop("AddItem", () => mx.AddItem(serverHandle, itemAddress));
+                node = CacheItem(mx, itemAddress, itemHandle);
+                int added = itemHandle;
                 // MXAccess rejects writes to added-but-unadvised items with E_INVALIDARG.
-                mx.Advise(serverHandle, itemHandle);
+                Interop("Advise", () => mx.Advise(serverHandle, added));
             }
 
             pumpStep();
-            step = "Write";
             baselineVersion = completionCache.CurrentVersion(serverHandle, itemHandle);
-            mx.Write(serverHandle, itemHandle, comment ?? string.Empty, userId);
+            int target = itemHandle;
+            Interop("Write", () => mx.Write(serverHandle, target, commentValue, userId));
         }
-        catch (COMException ex)
+        catch (AckInteropException ex)
         {
             int hresult = ex.HResult;
-            if (itemHandles.Remove(itemAddress))
+            if (itemHandles.TryGetValue(itemAddress, out LinkedListNode<AckItem>? failed))
             {
-                try { mx.UnAdvise(serverHandle, itemHandle); } catch { /* best effort */ }
+                DropItem(mx, failed!);
-                try { mx.RemoveItem(serverHandle, itemHandle); } catch { /* best effort */ }
             }
 
             if (IsServerGone(hresult))
@@ -149,7 +210,7 @@ public sealed class AlarmAckWriter : IAlarmAckWriter
             return AlarmAckResult.Create(
                 AlarmAckOutcome.WriteRejected,
                 hresult,
-                $"AckMsg {step} failed (HRESULT 0x{hresult:X8}).");
+                $"AckMsg {ex.Step} failed (HRESULT 0x{hresult:X8}).");
         }
 
         ulong baseline = baselineVersion;
@@ -162,6 +223,7 @@ public sealed class AlarmAckWriter : IAlarmAckWriter
                 pumpStep,
                 out RepeatedField<MxStatusProxy> statuses))
         {
+            node!.Value.Dirty = true;
             return AlarmAckResult.Create(
                 AlarmAckOutcome.Unconfirmed,
                 0,
@@ -192,6 +254,54 @@ public sealed class AlarmAckWriter : IAlarmAckWriter
         TearDownServer();
     }
 
+    /// <summary>
+    ///     Runs one MXAccess COM call and converts a failure the interop raised
+    ///     for its HRESULT into <see cref="AckInteropException"/>. Only the COM
+    ///     call itself runs inside the filter, so an exception thrown by this
+    ///     writer's own code (argument checks, caching, the pump) propagates as
+    ///     a bug instead of being reported as an MXAccess failure.
+    /// </summary>
+    /// <remarks>
+    ///     The runtime maps well-known HRESULTs to typed exceptions and the rest
+    ///     to COMException; each carries the HRESULT in Exception.HResult. The
+    ///     ArgumentException arm is needed because a wrong password or unknown
+    ///     user makes <c>AuthenticateUser</c> throw ArgumentException with
+    ///     E_INVALIDARG (0x80070057), observed live on windev on 2026-10-19.
+    ///     ArgumentNullException and ArgumentOutOfRangeException are excluded:
+    ///     the interop does not raise them for an HRESULT, so one escaping the
+    ///     call is an argument bug in the adapter, not an MXAccess failure.
+    /// </remarks>
+    private static T Interop<T>(string step, Func<T> call)
+    {
+        try
+        {
+            return call();
+        }
+        catch (Exception ex) when (IsInteropFailure(ex))
+        {
+            throw new AckInteropException(step, ex);
+        }
+    }
+
+    private static void Interop(string step, Action call)
+    {
+        try
+        {
+            call();
+        }
+        catch (Exception ex) when (IsInteropFailure(ex))
+        {
+            throw new AckInteropException(step, ex);
+        }
+    }
+
+    private static bool IsInteropFailure(Exception ex) =>
+        ex is ExternalException
+        || ex is UnauthorizedAccessException
+        || (ex is ArgumentException
+            && ex is not ArgumentNullException
+            && ex is not ArgumentOutOfRangeException);
+
     private static bool IsServerGone(int hresult) =>
         hresult == unchecked((int)0x80010108) // RPC_E_DISCONNECTED
         || hresult == unchecked((int)0x800706BA) // RPC_S_SERVER_UNAVAILABLE
@@ -208,16 +318,21 @@ public sealed class AlarmAckWriter : IAlarmAckWriter
         IMxAccessServer? mx = server;
         if (mx is not null)
         {
-            foreach (int itemHandle in itemHandles.Values)
+            foreach (AckItem item in itemLru)
             {
-                try { mx.UnAdvise(serverHandle, itemHandle); } catch { /* best effort */ }
+                try { mx.UnAdvise(serverHandle, item.Handle); } catch { /* best effort */ }
-                try { mx.RemoveItem(serverHandle, itemHandle); } catch { /* best effort */ }
+                try { mx.RemoveItem(serverHandle, item.Handle); } catch { /* best effort */ }
             }
 
             try { mx.Unregister(serverHandle); } catch { /* best effort */ }
         }
 
+        // The registration is gone, so no completion can arrive for its handles.
+        completionCache.RemoveServer(serverHandle);
+
         itemHandles.Clear();
+        itemLru.Clear();
+        heldItemHandles.Clear();
         server = null;
         comEventSource = null;
         object? toRelease = comObject;
@@ -228,6 +343,34 @@ public sealed class AlarmAckWriter : IAlarmAckWriter
         }
     }
 
+    /// <summary>Caches a newly added item as most recently used, evicting the least recently used past the cap.</summary>
+    private LinkedListNode<AckItem> CacheItem(IMxAccessServer mx, string itemAddress, int itemHandle)
+    {
+        while (itemLru.Count >= maxCachedItems && itemLru.Last is { } leastRecent)
+        {
+            DropItem(mx, leastRecent);
+        }
+
+        LinkedListNode<AckItem> node = itemLru.AddFirst(new AckItem(itemAddress, itemHandle));
+        itemHandles[itemAddress] = node;
+        heldItemHandles.Add(itemHandle);
+        return node;
+    }
+
+    /// <summary>Removes a cached item and best-effort releases its MXAccess item.</summary>
+    private void DropItem(IMxAccessServer mx, LinkedListNode<AckItem> node)
+    {
+        itemHandles.Remove(node.Value.Address);
+        itemLru.Remove(node);
+        heldItemHandles.Remove(node.Value.Handle);
+        try { mx.UnAdvise(serverHandle, node.Value.Handle); } catch { /* best effort */ }
+        try { mx.RemoveItem(serverHandle, node.Value.Handle); } catch { /* best effort */ }
+
+        // Without this the completion cache keeps one entry per dropped handle
+        // (each Unconfirmed re-add and each eviction) for the worker's life.
+        completionCache.Remove(serverHandle, node.Value.Handle);
+    }
+
     private IMxAccessServer EnsureRegistered()
     {
         if (server is not null) return server;
@@ -274,6 +417,53 @@ public sealed class AlarmAckWriter : IAlarmAckWriter
     {
         RepeatedField<MxStatusProxy> rows = new RepeatedField<MxStatusProxy>();
         rows.AddRange(statusConverter.ConvertMany(pVars));
-        completionCache.Record(hLMXServerHandle, phItemHandle, rows);
+        RecordWriteCompletion(hLMXServerHandle, phItemHandle, rows);
+    }
+
+    /// <summary>
+    ///     Records a write completion only for an item handle this writer
+    ///     currently holds on its live registration. A late completion for a
+    ///     dropped handle would otherwise recreate the cache entry that
+    ///     <see cref="DropItem"/> removed. Internal so tests can stand in for
+    ///     the COM event.
+    /// </summary>
+    internal void RecordWriteCompletion(int completionServerHandle, int itemHandle, RepeatedField<MxStatusProxy> rows)
+    {
+        if (server is null || completionServerHandle != serverHandle || !heldItemHandles.Contains(itemHandle))
+        {
+            return;
+        }
+
+        completionCache.Record(completionServerHandle, itemHandle, rows);
+    }
+
+    /// <summary>An MXAccess failure raised by one COM call, carrying that call's HRESULT.</summary>
+    private sealed class AckInteropException : Exception
+    {
+        public AckInteropException(string step, Exception inner)
+            : base("MXAccess " + step + " failed.", inner)
+        {
+            Step = step;
+            HResult = inner.HResult;
+        }
+
+        public string Step { get; }
+    }
+
+    /// <summary>A cached ack item. Plain class (not a record) for the net48 worker target.</summary>
+    private sealed class AckItem
+    {
+        public AckItem(string address, int handle)
+        {
+            Address = address;
+            Handle = handle;
+        }
+
+        public string Address { get; }
+
+        public int Handle { get; }
+
+        /// <summary>Set when the last write to this handle ended Unconfirmed; forces a fresh handle next time.</summary>
+        public bool Dirty { get; set; }
     }
 }

src/ZB.MOM.WW.MxGateway.Worker/MxAccess/AlarmCommandHandler.cs

@@ -41,6 +41,7 @@ public sealed class AlarmCommandHandler : IAlarmCommandHandler
     private readonly IMxAccessComObjectFactory? comFactory;
     private readonly Func<IReadOnlyList<AlarmSubtagTarget>, IMxAccessAlarmConsumer>? standbyFactory;
     private readonly Func<IAlarmAckWriter>? ackWriterFactory;
+    private readonly Action? ackPumpStep;
     private readonly MxAccessEventMapper mapper = new MxAccessEventMapper();
     private readonly object syncRoot = new object();
     private readonly IAlarmAckWriter deferredAckWriter;
@@ -131,13 +132,21 @@ public sealed class AlarmCommandHandler : IAlarmCommandHandler
     ///     unsubscribe/resubscribe, and is disposed with the handler. Tests
     ///     inject a fake.
     /// </param>
+    /// <param name="ackPumpStep">
+    ///     Pump step the default <see cref="AlarmAckWriter"/> drives while it waits
+    ///     for OnWriteComplete. Production passes the STA runtime's
+    ///     <c>StaRuntime.PumpPendingMessages</c>, which also refreshes the STA
+    ///     watchdog heartbeat; when <see langword="null"/> a bare
+    ///     <see cref="StaMessagePump"/> is used (no heartbeat refresh).
+    /// </param>
     public AlarmCommandHandler(
         MxAccessEventQueue eventQueue,
         Func<IMxAccessAlarmConsumer> consumerFactory,
         Action? threadAffinityCheck,
         IMxAccessComObjectFactory? comFactory,
         Func<IReadOnlyList<AlarmSubtagTarget>, IMxAccessAlarmConsumer>? standbyFactory,
-        Func<IAlarmAckWriter>? ackWriterFactory = null)
+        Func<IAlarmAckWriter>? ackWriterFactory = null,
+        Action? ackPumpStep = null)
     {
         this.eventQueue = eventQueue ?? throw new ArgumentNullException(nameof(eventQueue));
         this.consumerFactory = consumerFactory ?? throw new ArgumentNullException(nameof(consumerFactory));
@@ -145,6 +154,7 @@ public sealed class AlarmCommandHandler : IAlarmCommandHandler
         this.comFactory = comFactory;
         this.standbyFactory = standbyFactory;
         this.ackWriterFactory = ackWriterFactory;
+        this.ackPumpStep = ackPumpStep;
         this.deferredAckWriter = new DeferredAckWriter(this);
     }
 
@@ -236,13 +246,32 @@ public sealed class AlarmCommandHandler : IAlarmCommandHandler
         {
             IMxAccessAlarmConsumer primary = consumerFactory()
                 ?? throw new InvalidOperationException("Alarm consumer factory returned null.");
-            IMxAccessAlarmConsumer standby = BuildStandby(watchList);
+            IMxAccessAlarmConsumer? standby = null;
-            AlarmFailoverConfig? failoverConfig = command.Failover;
+            try
-            FailoverSettings settings = new FailoverSettings(
+            {
-                failoverConfig?.ConsecutiveFailureThreshold ?? 3,
+                standby = BuildStandby(watchList);
-                failoverConfig?.FailbackProbeIntervalSeconds ?? 30,
+                AlarmFailoverConfig? failoverConfig = command.Failover;
-                failoverConfig?.FailbackStableProbes ?? 3);
+                FailoverSettings settings = new FailoverSettings(
-            return new FailoverAlarmConsumer(primary, standby, settings);
+                    failoverConfig?.ConsecutiveFailureThreshold ?? 3,
+                    failoverConfig?.FailbackProbeIntervalSeconds ?? 30,
+                    failoverConfig?.FailbackStableProbes ?? 3);
+                return new FailoverAlarmConsumer(primary, standby, settings);
+            }
+            catch
+            {
+                // The primary (and a standby that was built before the
+                // composite constructor threw) is owned by nothing yet: the
+                // failed Subscribe never reaches the dispatcher that would
+                // dispose it, so release it here or its wnwrap COM object
+                // leaks for the life of the worker.
+                if (standby is not null)
+                {
+                    try { standby.Dispose(); } catch { /* swallow — preserve the build failure */ }
+                }
+
+                try { primary.Dispose(); } catch { /* swallow — preserve the build failure */ }
+                throw;
+            }
         }
 
         // Alarmmgr, or Unspecified with an empty watch list — primary only.
@@ -399,12 +428,14 @@ public sealed class AlarmCommandHandler : IAlarmCommandHandler
                     "Alarm acknowledgement requires an IMxAccessComObjectFactory; the alarm command handler was constructed without one.");
             }
 
-            // StaMessagePump.PumpPendingMessages is PeekMessage/DispatchMessage on
+            // The pump step is PeekMessage/DispatchMessage on the calling thread —
-            // the calling thread — the STA, since acks run there — which is what
+            // the STA, since acks run there — which is what delivers the writer's
-            // delivers the writer's OnWriteComplete callback.
+            // OnWriteComplete callback. The injected StaRuntime pump also marks STA
+            // activity so a long completion wait does not read as a hung STA.
+            Action pump = ackPumpStep ?? (() => new StaMessagePump().PumpPendingMessages());
             ackWriter = new AlarmAckWriter(
                 comFactory,
-                () => new StaMessagePump().PumpPendingMessages(),
+                pump,
                 MxAccessStaSession.ResolveWriteCompletionTimeout(),
                 AlarmAckWriter.ResolveClientName());
             return ackWriter;

src/ZB.MOM.WW.MxGateway.Worker/MxAccess/AlarmRecordTransitionMapper.cs

@@ -49,6 +49,17 @@ public static class AlarmRecordTransitionMapper
     ///         <item><description><c>previous=*Rtn</c> + <c>current=*Alm</c> → Raise (re-trigger after clear).</description></item>
     ///         <item><description>Anything else → Unspecified (no proto kind to emit).</description></item>
     ///     </list>
+    ///     <para>
+    ///         <see cref="AlarmTransitionKind.Retrigger"/> is never returned. It is
+    ///         reserved in the contract, but neither alarm source gives the worker
+    ///         a native re-alarm signal: the alarmmgr XML carries only the four
+    ///         <c>*_ALM</c>/<c>*_RTN</c> states, so a return-to-alarm is
+    ///         indistinguishable from a fresh raise and is reported as Raise, and
+    ///         the subtag source observes no re-alarm counter. Deliberately not
+    ///         synthesized (the gateway forwards only transitions the source
+    ///         reports — see docs/AlarmClientDiscovery.md, "Retrigger
+    ///         transitions").
+    ///     </para>
     /// </summary>
     /// <param name="previous">The previous alarm state kind.</param>
     /// <param name="current">The current alarm state kind.</param>

src/ZB.MOM.WW.MxGateway.Worker/MxAccess/MxAccessSession.cs

@@ -222,6 +222,13 @@ public sealed class MxAccessSession : IDisposable
 
         mxAccessServer.Unregister(serverHandle);
         handleRegistry.UnregisterServerHandle(serverHandle);
+        // Unregister drops every item under the server in one call, so the
+        // per-item evictions in RemoveItem never run for them. Evict both
+        // per-handle caches by server so a long-lived worker that registers
+        // and unregisters clients does not accumulate one entry per item it
+        // ever wrote or read, and a reused server handle id starts clean.
+        valueCache.RemoveServer(serverHandle);
+        writeCompletionCache.RemoveServer(serverHandle);
     }
 
     /// <summary>Adds an item to an MXAccess server and returns the item handle.</summary>
@@ -235,6 +242,7 @@ public sealed class MxAccessSession : IDisposable
         ThrowIfDisposed();
 
         int itemHandle = mxAccessServer.AddItem(serverHandle, itemDefinition);
+        EvictStaleHandleState(serverHandle, itemHandle);
         handleRegistry.RegisterItemHandle(
             serverHandle,
             itemHandle,
@@ -258,6 +266,7 @@ public sealed class MxAccessSession : IDisposable
         ThrowIfDisposed();
 
         int itemHandle = mxAccessServer.AddItem2(serverHandle, itemDefinition, itemContext);
+        EvictStaleHandleState(serverHandle, itemHandle);
         handleRegistry.RegisterItemHandle(
             serverHandle,
             itemHandle,
@@ -268,6 +277,30 @@ public sealed class MxAccessSession : IDisposable
         return itemHandle;
     }
 
+    /// <summary>
+    ///     Drops cached per-handle state left over from a previous lifetime of
+    ///     a handle id MXAccess just handed out again. RemoveItem and Unregister
+    ///     already evict, but an OnDataChange or OnWriteComplete that was
+    ///     queued on the STA when they ran is dispatched afterwards (the loop
+    ///     pumps between work items) and recreates the entry. Without this, a
+    ///     reused id would let ReadBulk's cached fast path serve that stale
+    ///     value as <c>was_cached = true</c>. A handle the registry still
+    ///     tracks is a live item (MXAccess returned an existing handle), so its
+    ///     current entries are kept.
+    /// </summary>
+    private void EvictStaleHandleState(
+        int serverHandle,
+        int itemHandle)
+    {
+        if (handleRegistry.ContainsItemHandle(serverHandle, itemHandle))
+        {
+            return;
+        }
+
+        valueCache.Remove(serverHandle, itemHandle);
+        writeCompletionCache.Remove(serverHandle, itemHandle);
+    }
+
     /// <summary>Removes an item from an MXAccess server.</summary>
     /// <param name="serverHandle">Handle returned by the worker.</param>
     /// <param name="itemHandle">Handle returned by the worker.</param>
@@ -283,6 +316,11 @@ public sealed class MxAccessSession : IDisposable
         // same handle id (which MXAccess may reuse) does not serve a stale
         // OnDataChange snapshot from the previous lifetime.
         valueCache.Remove(serverHandle, itemHandle);
+        // Same for the OnWriteComplete completion entry: the event sink records
+        // one per written handle pair, so without this an AddItem -> Write ->
+        // RemoveItem churn grows the cache by one entry per cycle for the life
+        // of the worker.
+        writeCompletionCache.Remove(serverHandle, itemHandle);
     }
 
     /// <summary>Advises on item changes with plain subscription.</summary>
@@ -397,6 +435,7 @@ public sealed class MxAccessSession : IDisposable
         ThrowIfDisposed();
 
         int itemHandle = mxAccessServer.AddBufferedItem(serverHandle, itemDefinition, itemContext);
+        EvictStaleHandleState(serverHandle, itemHandle);
         handleRegistry.RegisterItemHandle(
             serverHandle,
             itemHandle,
@@ -870,14 +909,28 @@ public sealed class MxAccessSession : IDisposable
     ///     OnDataChange value if the tag is already advised AND a cached value
     ///     exists (no subscription side effects); otherwise takes the AddItem
     ///     + Advise + wait + UnAdvise + RemoveItem snapshot lifecycle itself.
-    ///     <paramref name="timeout"/> bounds the wait per-tag in the snapshot
+    ///     <paramref name="timeout"/> bounds the wait for every snapshot tag;
-    ///     case; <paramref name="pumpStep"/> is invoked on every poll
+    ///     <paramref name="pumpStep"/> is invoked on every poll iteration so the
-    ///     iteration so the worker's STA can dispatch the incoming MXAccess
+    ///     worker's STA can dispatch the incoming MXAccess message that carries
-    ///     message that carries the value.
+    ///     the value.
     /// </summary>
+    /// <remarks>
+    ///     The snapshot tags are processed in phases rather than one at a time:
+    ///     every cache probe runs first, then every uncached tag is added and
+    ///     advised, then one wait with a single shared deadline collects the
+    ///     first OnDataChange per handle, then every snapshot handle is torn
+    ///     down. A serial lifecycle held the STA for up to N x timeout when N
+    ///     tags never produced a value; the phased form holds it for at most
+    ///     one timeout. Each tag's result is still its own: a tag that never
+    ///     delivered fails with the same timed-out diagnostic and its own item
+    ///     handle, an AddItem or Advise failure fails only that tag, and results
+    ///     keep request order. The cache probes run before any snapshot AddItem
+    ///     so a later tag can never be served from an earlier tag's own
+    ///     snapshot subscription.
+    /// </remarks>
     /// <param name="serverHandle">The MXAccess server handle.</param>
     /// <param name="tagAddresses">The tag addresses to read.</param>
-    /// <param name="timeout">The timeout per tag.</param>
+    /// <param name="timeout">The timeout bounding the shared snapshot wait.</param>
     /// <param name="pumpStep">Action invoked on each poll iteration.</param>
     /// <returns>The per-tag read results, in the order the tags were provided.</returns>
     public IReadOnlyList<BulkReadResult> ReadBulk(
@@ -897,91 +950,225 @@ public sealed class MxAccessSession : IDisposable
             throw new ArgumentNullException(nameof(pumpStep));
         }
 
-        List<BulkReadResult> results = new(tagAddresses.Count);
+        int tagCount = tagAddresses.Count;
-        foreach (string? tagAddress in tagAddresses)
+        BulkReadResult?[] results = new BulkReadResult?[tagCount];
+        List<SnapshotRead> snapshots = new();
+
+        // Phase 1: validation and the cached-and-advised fast path. A tag
+        // already advised by the caller with a cached value is answered without
+        // touching the existing subscription — the caller didn't create it, so
+        // ReadBulk must not tear it down.
+        for (int index = 0; index < tagCount; index++)
         {
+            string? tagAddress = tagAddresses[index];
             if (string.IsNullOrWhiteSpace(tagAddress))
             {
-                results.Add(FailedRead(serverHandle, tagAddress ?? string.Empty, itemHandle: 0, wasCached: false, "Tag address is required."));
+                results[index] = FailedRead(serverHandle, tagAddress ?? string.Empty, itemHandle: 0, wasCached: false, "Tag address is required.");
                 continue;
             }
 
-            results.Add(ReadOneTag(serverHandle, tagAddress, timeout, pumpStep));
+            if (TryGetCachedReadFor(serverHandle, tagAddress!, out int cachedItemHandle, out MxAccessValueCache.CachedValue cachedValue))
+            {
+                results[index] = SucceededRead(serverHandle, tagAddress!, cachedItemHandle, wasCached: true, cachedValue);
+                continue;
+            }
+
+            snapshots.Add(new SnapshotRead(index, tagAddress!));
         }
 
-        return results;
+        if (snapshots.Count > 0)
+        {
+            try
+            {
+                ReadSnapshots(serverHandle, snapshots, results, timeout, pumpStep);
+            }
+            finally
+            {
+                TearDownSnapshots(serverHandle, snapshots, pumpStep);
+            }
+        }
+
+        List<BulkReadResult> ordered = new(tagCount);
+        for (int index = 0; index < tagCount; index++)
+        {
+            ordered.Add(results[index]!);
+        }
+
+        return ordered;
     }
 
-    private BulkReadResult ReadOneTag(
+    /// <summary>
+    ///     Number of snapshot items ReadBulk adds/advises (phase 2) or tears
+    ///     down (phase 4) between pump steps. Those phases make one or two COM
+    ///     calls per tag without waiting, so a large batch would otherwise hold
+    ///     the STA with no pump and no activity mark for its whole length; a
+    ///     pump every <see cref="ReadBulkPumpInterval"/> items keeps inbound
+    ///     callbacks flowing and keeps the STA heartbeat fresh, so a big batch
+    ///     cannot drift toward <c>HeartbeatStuckCeiling</c>.
+    /// </summary>
+    internal const int ReadBulkPumpInterval = 16;
+
+    private void ReadSnapshots(
         int serverHandle,
-        string tagAddress,
+        List<SnapshotRead> snapshots,
+        BulkReadResult?[] results,
         TimeSpan timeout,
         Action pumpStep)
     {
-        // 1. Cached-and-advised fast path: scan the registry for a live item
+        // Phase 2: reserve our own item handle and advise for every uncached
-        //    handle matching this tag and check whether the value cache has a
+        // tag. A per-tag AddItem/Advise failure fails only that tag. When
-        //    payload for it. If so, return the cached value without touching
+        // MXAccess hands back a handle an earlier tag in this batch already
-        //    the existing subscription — the caller didn't create it, so
+        // advised (a duplicate address), that tag shares the earlier tag's
-        //    ReadBulk must not tear it down.
+        // subscription instead of advising the handle a second time, and gets
-        if (TryGetCachedReadFor(serverHandle, tagAddress, out int cachedItemHandle, out MxAccessValueCache.CachedValue cachedValue))
+        // the same outcome fanned out to it after the wait.
+        List<SnapshotRead> waiting = new(snapshots.Count);
+        Dictionary<int, SnapshotRead> advisedByHandle = new();
+        List<SnapshotRead> sharing = new();
+        for (int position = 0; position < snapshots.Count; position++)
         {
-            return SucceededRead(
+            SnapshotRead snapshot = snapshots[position];
-                serverHandle,
+            try
-                tagAddress,
+            {
-                cachedItemHandle,
+                if (position > 0 && position % ReadBulkPumpInterval == 0)
-                wasCached: true,
+                {
-                cachedValue);
+                    pumpStep();
+                }
+
+                snapshot.ItemHandle = AddItem(serverHandle, snapshot.TagAddress);
+                if (advisedByHandle.TryGetValue(snapshot.ItemHandle, out SnapshotRead? owner))
+                {
+                    snapshot.SharedWith = owner;
+                    sharing.Add(snapshot);
+                    continue;
+                }
+
+                snapshot.Baseline = valueCache.CurrentVersion(serverHandle, snapshot.ItemHandle);
+                Advise(serverHandle, snapshot.ItemHandle);
+                snapshot.Advised = true;
+                advisedByHandle[snapshot.ItemHandle] = snapshot;
+                waiting.Add(snapshot);
+            }
+            catch (Exception exception)
+            {
+                results[snapshot.Index] = FailedRead(serverHandle, snapshot.TagAddress, snapshot.ItemHandle, wasCached: false, exception.Message);
+            }
         }
 
-        // 2. Snapshot lifecycle. Reserve our own item handle, advise, pump
+        if (waiting.Count > 0)
-        //    until we see a fresh OnDataChange (or the deadline elapses),
+        {
-        //    then tear it down.
+            WaitForSnapshots(serverHandle, waiting, results, timeout, pumpStep);
-        int itemHandle = 0;
+        }
-        bool advised = false;
+
+        foreach (SnapshotRead duplicate in sharing)
+        {
+            BulkReadResult? ownerResult = results[duplicate.SharedWith!.Index];
+            if (ownerResult is null)
+            {
+                continue;
+            }
+
+            BulkReadResult fanned = ownerResult.Clone();
+            fanned.TagAddress = duplicate.TagAddress;
+            results[duplicate.Index] = fanned;
+        }
+    }
+
+    private void WaitForSnapshots(
+        int serverHandle,
+        List<SnapshotRead> waiting,
+        BulkReadResult?[] results,
+        TimeSpan timeout,
+        Action pumpStep)
+    {
+        // Phase 3: one shared deadline. Pump until every advised handle has
+        // seen its first OnDataChange since its baseline, or the deadline
+        // elapses.
+        int[] itemHandles = new int[waiting.Count];
+        ulong[] baselines = new ulong[waiting.Count];
+        for (int index = 0; index < waiting.Count; index++)
+        {
+            itemHandles[index] = waiting[index].ItemHandle;
+            baselines[index] = waiting[index].Baseline;
+        }
+
+        bool[] received = new bool[waiting.Count];
+        MxAccessValueCache.CachedValue[] values = new MxAccessValueCache.CachedValue[waiting.Count];
         try
         {
-            itemHandle = AddItem(serverHandle, tagAddress);
-            ulong baseline = valueCache.CurrentVersion(serverHandle, itemHandle);
-            Advise(serverHandle, itemHandle);
-            advised = true;
-
             DateTime deadline = DateTime.UtcNow + timeout;
-            bool gotValue = valueCache.TryWaitForUpdate(
+            valueCache.TryWaitForUpdates(serverHandle, itemHandles, baselines, deadline, pumpStep, received, values);
-                serverHandle,
-                itemHandle,
-                baseline,
-                deadline,
-                pumpStep,
-                out MxAccessValueCache.CachedValue snapshot);
-
-            return gotValue
-                ? SucceededRead(serverHandle, tagAddress, itemHandle, wasCached: false, snapshot)
-                : FailedRead(
-                    serverHandle,
-                    tagAddress,
-                    itemHandle,
-                    wasCached: false,
-                    $"ReadBulk timed out after {timeout.TotalMilliseconds:F0} ms waiting for first OnDataChange.");
         }
         catch (Exception exception)
         {
-            return FailedRead(serverHandle, tagAddress, itemHandle, wasCached: false, exception.Message);
+            // A pump failure ends the wait for every tag still outstanding;
-        }
+            // tags captured before it keep their values.
-        finally
+            for (int index = 0; index < waiting.Count; index++)
-        {
-            // Snapshot teardown — best-effort. Errors here are noted on the
-            // diagnostic message of the original result (above) by appending
-            // a cleanup suffix; we never re-throw from finally.
-            if (advised)
             {
-                try { UnAdvise(serverHandle, itemHandle); } catch { /* swallow — best effort */ }
+                if (!received[index])
+                {
+                    SnapshotRead failed = waiting[index];
+                    results[failed.Index] = FailedRead(serverHandle, failed.TagAddress, failed.ItemHandle, wasCached: false, exception.Message);
+                }
+            }
+        }
+
+        for (int index = 0; index < waiting.Count; index++)
+        {
+            SnapshotRead snapshot = waiting[index];
+            if (results[snapshot.Index] is not null)
+            {
+                continue;
             }
 
-            if (itemHandle != 0)
+            results[snapshot.Index] = received[index]
+                ? SucceededRead(serverHandle, snapshot.TagAddress, snapshot.ItemHandle, wasCached: false, values[index])
+                : FailedRead(
+                    serverHandle,
+                    snapshot.TagAddress,
+                    snapshot.ItemHandle,
+                    wasCached: false,
+                    $"ReadBulk timed out after {timeout.TotalMilliseconds:F0} ms waiting for first OnDataChange.");
+        }
+    }
+
+    private void TearDownSnapshots(
+        int serverHandle,
+        List<SnapshotRead> snapshots,
+        Action pumpStep)
+    {
+        // Phase 4: snapshot teardown — best-effort, never re-throws. Each
+        // distinct handle is unadvised and removed exactly once, even when
+        // MXAccess handed the same id to several tags in the batch.
+        HashSet<int> unadvised = new();
+        HashSet<int> removed = new();
+        int calls = 0;
+        foreach (SnapshotRead snapshot in snapshots)
+        {
+            if (snapshot.Advised && unadvised.Add(snapshot.ItemHandle))
             {
-                try { RemoveItem(serverHandle, itemHandle); } catch { /* swallow — best effort */ }
+                PumpEvery(ref calls, pumpStep);
+                try { UnAdvise(serverHandle, snapshot.ItemHandle); } catch { /* swallow — best effort */ }
             }
         }
+
+        foreach (SnapshotRead snapshot in snapshots)
+        {
+            if (snapshot.ItemHandle != 0 && removed.Add(snapshot.ItemHandle))
+            {
+                PumpEvery(ref calls, pumpStep);
+                try { RemoveItem(serverHandle, snapshot.ItemHandle); } catch { /* swallow — best effort */ }
+            }
+        }
+    }
+
+    private static void PumpEvery(ref int calls, Action pumpStep)
+    {
+        if (calls > 0 && calls % ReadBulkPumpInterval == 0)
+        {
+            try { pumpStep(); } catch { /* swallow — teardown is best effort */ }
+        }
+
+        calls++;
     }
 
     private bool TryGetCachedReadFor(
@@ -1308,6 +1495,41 @@ public sealed class MxAccessSession : IDisposable
         }
     }
 
+    /// <summary>
+    ///     Mutable per-tag state for one ReadBulk snapshot lifecycle. A plain
+    ///     class with get/set properties (no records or init accessors) so it
+    ///     compiles under the worker's net48 target.
+    /// </summary>
+    private sealed class SnapshotRead
+    {
+        public SnapshotRead(int index, string tagAddress)
+        {
+            Index = index;
+            TagAddress = tagAddress;
+        }
+
+        /// <summary>Gets the position of the tag in the request.</summary>
+        public int Index { get; }
+
+        /// <summary>Gets the requested tag address.</summary>
+        public string TagAddress { get; }
+
+        /// <summary>Gets or sets the snapshot item handle, 0 until AddItem succeeds.</summary>
+        public int ItemHandle { get; set; }
+
+        /// <summary>Gets or sets the value-cache version captured before Advise.</summary>
+        public ulong Baseline { get; set; }
+
+        /// <summary>Gets or sets a value indicating whether Advise succeeded.</summary>
+        public bool Advised { get; set; }
+
+        /// <summary>
+        ///     Gets or sets the earlier snapshot in the same batch whose
+        ///     subscription this one shares (same item handle), or null.
+        /// </summary>
+        public SnapshotRead? SharedWith { get; set; }
+    }
+
     private void ThrowIfDisposed()
     {
         if (disposed)

src/ZB.MOM.WW.MxGateway.Worker/MxAccess/MxAccessStaSession.cs

@@ -42,13 +42,24 @@ public sealed class MxAccessStaSession : IWorkerRuntimeSession
     /// <summary>Default alarm poll cadence when the environment says nothing usable.</summary>
     internal static readonly TimeSpan DefaultAlarmPollInterval = TimeSpan.FromMilliseconds(500);
 
-    private readonly TimeSpan alarmPollInterval = ResolveAlarmPollInterval();
+    /// <summary>
+    ///     Consecutive alarm-poll failures that fault the session. Matches the
+    ///     default of <see cref="FailoverSettings.Threshold"/>
+    ///     (<c>AlarmFailoverConfig.consecutive_failure_threshold</c>, default 3):
+    ///     one transient COM hiccup in <c>PollOnce</c> is logged and retried on
+    ///     the next tick, and only a run of failures with no success in between
+    ///     is treated as a dead subscription.
+    /// </summary>
+    internal const int DefaultAlarmPollFailureThreshold = 3;
 
     private readonly IMxAccessComObjectFactory factory;
     private readonly IMxAccessEventSink eventSink;
     private readonly MxAccessEventQueue eventQueue;
     private readonly StaRuntime staRuntime;
-    private readonly Func<MxAccessEventQueue, Action, IMxAccessComObjectFactory, IAlarmCommandHandler>? alarmCommandHandlerFactory;
+    // Pump-aware form: the fourth argument is the STA runtime's pump step, so
+    // handler work that pumps while it waits (the alarm ack writer's
+    // OnWriteComplete wait) also refreshes the STA watchdog heartbeat.
+    private readonly Func<MxAccessEventQueue, Action, IMxAccessComObjectFactory, Action, IAlarmCommandHandler>? alarmCommandHandlerFactory;
     private StaCommandDispatcher? commandDispatcher;
     private MxAccessSession? session;
     private IAlarmCommandHandler? alarmCommandHandler;
@@ -96,6 +107,23 @@ public sealed class MxAccessStaSession : IWorkerRuntimeSession
     {
     }
 
+    /// <summary>
+    /// Initializes a new instance of <see cref="MxAccessStaSession"/> with default dependencies and a
+    /// pump-aware alarm-command handler factory. Its fourth argument is a pump step bound to this
+    /// session's <see cref="StaRuntime"/> (<see cref="StaRuntime.PumpPendingMessages"/>), which
+    /// refreshes the STA watchdog heartbeat on every pump.
+    /// </summary>
+    /// <param name="pumpAwareAlarmCommandHandlerFactory">Factory that constructs the alarm-command handler.</param>
+    internal MxAccessStaSession(Func<MxAccessEventQueue, Action, IMxAccessComObjectFactory, Action, IAlarmCommandHandler> pumpAwareAlarmCommandHandlerFactory)
+    {
+        staRuntime = new StaRuntime();
+        factory = new MxAccessComObjectFactory();
+        eventQueue = new MxAccessEventQueue(MxAccessEventQueue.ResolveCapacity());
+        eventSink = new MxAccessBaseEventSink(eventQueue);
+        alarmCommandHandlerFactory = pumpAwareAlarmCommandHandlerFactory
+            ?? throw new ArgumentNullException(nameof(pumpAwareAlarmCommandHandlerFactory));
+    }
+
     /// <summary>
     /// Initializes a new instance of <see cref="MxAccessStaSession"/> with custom STA runtime and factory.
     /// </summary>
@@ -183,7 +211,9 @@ public sealed class MxAccessStaSession : IWorkerRuntimeSession
         this.factory = factory ?? throw new ArgumentNullException(nameof(factory));
         this.eventSink = eventSink ?? throw new ArgumentNullException(nameof(eventSink));
         this.eventQueue = eventQueue ?? throw new ArgumentNullException(nameof(eventQueue));
-        this.alarmCommandHandlerFactory = alarmCommandHandlerFactory;
+        this.alarmCommandHandlerFactory = alarmCommandHandlerFactory is null
+            ? null
+            : (eq, affinity, comFactory, _) => alarmCommandHandlerFactory(eq, affinity, comFactory);
     }
 
     /// <summary>
@@ -199,6 +229,28 @@ public sealed class MxAccessStaSession : IWorkerRuntimeSession
     /// </summary>
     internal TimeSpan WriteCompletionTimeout { get; set; } = ResolveWriteCompletionTimeout();
 
+    /// <summary>
+    ///     Cadence the alarm poll loop waits between polls. Resolved from the
+    ///     launcher environment; internal-settable as a test seam (set before
+    ///     <see cref="StartAsync(string, int, CancellationToken)"/>).
+    /// </summary>
+    internal TimeSpan AlarmPollInterval { get; set; } = ResolveAlarmPollInterval();
+
+    /// <summary>
+    ///     Consecutive alarm-poll failures that fault the session; clamped to a
+    ///     minimum of 1. Internal-settable as a test seam.
+    /// </summary>
+    internal int AlarmPollFailureThreshold { get; set; } = DefaultAlarmPollFailureThreshold;
+
+    /// <summary>
+    ///     Sink for the transient alarm-poll failure warning. Defaults to
+    ///     <see cref="Console.Error"/>, the stream <c>WorkerConsoleLogger</c>
+    ///     writes to. The line carries the exception type, HRESULT, and
+    ///     failure counts only — never the exception message, which can echo
+    ///     alarm or tag names. Tests substitute a collector.
+    /// </summary>
+    internal Action<string>? AlarmPollWarningSink { get; set; }
+
     /// <summary>
     ///     Resolves the write-completion wait from the launcher-provided
     ///     environment variable; a missing or invalid value falls back to
@@ -266,6 +318,13 @@ public sealed class MxAccessStaSession : IWorkerRuntimeSession
     {
         staRuntime.Start();
 
+        // WRK-02: a post-start STA loop death has no other observer. Record it
+        // on the event queue so the IPC drain loop sends a WorkerFault to the
+        // gateway and ends the pipe session (and with it the worker process),
+        // instead of leaving a live process whose commands queue forever.
+        staRuntime.RegisterLoopFaultHandler(
+            exception => eventQueue.RecordFault(CreateStaLoopFault(exception)));
+
         WorkerReady ready = await staRuntime.InvokeAsync(
             () =>
             {
@@ -288,7 +347,8 @@ public sealed class MxAccessStaSession : IWorkerRuntimeSession
                     alarmCommandHandler = alarmCommandHandlerFactory(
                         eventQueue,
                         EnsureOnAlarmConsumerThread,
-                        factory);
+                        factory,
+                        () => staRuntime.PumpPendingMessages());
                 }
                 commandDispatcher = new StaCommandDispatcher(
                     staRuntime,
@@ -322,13 +382,16 @@ public sealed class MxAccessStaSession : IWorkerRuntimeSession
         IAlarmCommandHandler handler,
         CancellationToken cancellationToken)
     {
+        TimeSpan pollInterval = AlarmPollInterval;
+        int failureThreshold = AlarmPollFailureThreshold < 1 ? 1 : AlarmPollFailureThreshold;
         return Task.Run(async () =>
         {
+            int consecutiveFailures = 0;
             while (!cancellationToken.IsCancellationRequested)
             {
                 try
                 {
-                    await Task.Delay(alarmPollInterval, cancellationToken).ConfigureAwait(false);
+                    await Task.Delay(pollInterval, cancellationToken).ConfigureAwait(false);
                 }
                 catch (OperationCanceledException)
                 {
@@ -361,6 +424,7 @@ public sealed class MxAccessStaSession : IWorkerRuntimeSession
                             }
                         },
                         cancellationToken).ConfigureAwait(false);
+                    consecutiveFailures = 0;
                 }
                 catch (OperationCanceledException)
                 {
@@ -390,19 +454,66 @@ public sealed class MxAccessStaSession : IWorkerRuntimeSession
                     // A real alarm-poll failure (COMException from
                     // GetXmlCurrentAlarms2, malformed-XML parse failure, an
                     // STA-affinity InvalidOperationException from
-                    // EnsureOnAlarmConsumerThread, etc.). Record it as a
+                    // EnsureOnAlarmConsumerThread, etc.). A single failure is
-                    // fault on the event queue so a broken alarm subscription
+                    // logged and retried on the next tick: one transient COM
-                    // — or an affinity-invariant regression — becomes
+                    // hiccup must not tear down the whole session (every data
+                    // subscription with it). Only a run of
+                    // AlarmPollFailureThreshold failures with no success in
+                    // between is recorded as a fault on the event queue, so a
+                    // broken alarm subscription — or an affinity-invariant
+                    // regression, which fails every tick — still becomes
                     // observable on the IPC fault path instead of silently
                     // faulting this never-awaited task. The loop then stops —
                     // the subscription is dead.
-                    eventQueue.RecordFault(CreateAlarmPollFault(exception));
+                    if (staRuntime.LoopFault is not null)
-                    return;
+                    {
+                        // The STA loop itself died; its handler already recorded
+                        // the StaHung fault. This failure is just the fast-fail
+                        // InvokeAsync returns afterwards — recording it would add
+                        // a second, mislabelled alarm-poll fault.
+                        return;
+                    }
+
+                    consecutiveFailures++;
+                    if (consecutiveFailures >= failureThreshold)
+                    {
+                        eventQueue.RecordFault(CreateAlarmPollFault(exception));
+                        return;
+                    }
+
+                    WriteAlarmPollWarning(exception, consecutiveFailures, failureThreshold);
                 }
             }
         }, CancellationToken.None);
     }
 
+    private void WriteAlarmPollWarning(Exception exception, int consecutiveFailures, int failureThreshold)
+    {
+        // Same "level=<L> event=<E> k=v" line shape as WorkerConsoleLogger and
+        // the wnwrap truncation warning. Type and HRESULT only — the exception
+        // message is deliberately left out (it can carry alarm or tag names).
+        string hresult = exception is System.Runtime.InteropServices.COMException comException
+            ? "0x" + comException.HResult.ToString("X8", System.Globalization.CultureInfo.InvariantCulture)
+            : "none";
+        string message = string.Format(
+            System.Globalization.CultureInfo.InvariantCulture,
+            "level=Warning event=AlarmPollFailed exceptionType={0} hresult={1} consecutiveFailures={2} faultThreshold={3}",
+            exception.GetType().FullName,
+            hresult,
+            consecutiveFailures,
+            failureThreshold);
+
+        try
+        {
+            Action<string> sink = AlarmPollWarningSink ?? Console.Error.WriteLine;
+            sink(message);
+        }
+        catch
+        {
+            // Logging must never end the poll loop.
+        }
+    }
+
     private void EnsureOnAlarmConsumerThread()
     {
         AssertOnAlarmConsumerThread(alarmConsumerThreadId, Environment.CurrentManagedThreadId);
@@ -435,6 +546,41 @@ public sealed class MxAccessStaSession : IWorkerRuntimeSession
         }
     }
 
+    /// <summary>
+    ///     Builds the fault reported when the STA loop dies after start. Uses
+    ///     <see cref="WorkerFaultCategory.StaHung"/>: the gateway already treats
+    ///     that category as "this worker's apartment can no longer run COM
+    ///     work", which is exactly the state a dead loop leaves behind.
+    /// </summary>
+    internal static WorkerFault CreateStaLoopFault(Exception exception)
+    {
+        // Type and HRESULT only, never exception.Message — the same type-only
+        // rule every other worker failure line follows.
+        string message = string.Format(
+            System.Globalization.CultureInfo.InvariantCulture,
+            "Worker STA loop terminated unexpectedly: exceptionType={0} hresult=0x{1:X8}",
+            exception.GetType().FullName,
+            exception.HResult);
+        WorkerFault fault = new()
+        {
+            Category = WorkerFaultCategory.StaHung,
+            ExceptionType = exception.GetType().FullName ?? string.Empty,
+            DiagnosticMessage = message,
+            ProtocolStatus = new ProtocolStatus
+            {
+                Code = ProtocolStatusCode.WorkerUnavailable,
+                Message = message,
+            },
+        };
+
+        if (exception is System.Runtime.InteropServices.COMException comException)
+        {
+            fault.Hresult = comException.HResult;
+        }
+
+        return fault;
+    }
+
     private static WorkerFault CreateAlarmPollFault(Exception exception)
     {
         string message =

src/ZB.MOM.WW.MxGateway.Worker/MxAccess/MxAccessValueCache.cs

@@ -162,6 +162,46 @@ public sealed class MxAccessValueCache
         }
     }
 
+    /// <summary>
+    ///     Removes every cache slot recorded under a server handle. The session
+    ///     calls this when a server handle is unregistered: MXAccess drops all of
+    ///     that server's items with it, so the per-item <see cref="Remove"/>
+    ///     never runs for them and their entries would otherwise outlive the
+    ///     handle (and could be served to a later server that reuses the id).
+    /// </summary>
+    /// <param name="serverHandle">MXAccess server handle that was unregistered.</param>
+    public void RemoveServer(int serverHandle)
+    {
+        lock (syncRoot)
+        {
+            List<long> stale = new();
+            foreach (long key in entries.Keys)
+            {
+                if ((int)(key >> 32) == serverHandle)
+                {
+                    stale.Add(key);
+                }
+            }
+
+            foreach (long key in stale)
+            {
+                entries.Remove(key);
+            }
+        }
+    }
+
+    /// <summary>Gets the number of handle pairs with a cached value (test seam).</summary>
+    internal int Count
+    {
+        get
+        {
+            lock (syncRoot)
+            {
+                return entries.Count;
+            }
+        }
+    }
+
     /// <summary>
     ///     Waits until the cache entry's version exceeds <paramref name="sinceVersion"/>
     ///     or the deadline elapses, calling <paramref name="pumpStep"/> on every
@@ -231,6 +271,117 @@ public sealed class MxAccessValueCache
         }
     }
 
+    /// <summary>
+    ///     Multi-handle form of <see cref="TryWaitForUpdate"/>: waits until every
+    ///     handle in <paramref name="itemHandles"/> has an entry newer than its
+    ///     matching <paramref name="sinceVersions"/> slot, or the single shared
+    ///     deadline elapses. Each handle is captured the first time an update is
+    ///     observed for it (its first OnDataChange since the baseline), exactly
+    ///     as a dedicated per-handle wait would have captured it, so a bulk
+    ///     snapshot read waits at most one timeout in total rather than one per
+    ///     tag.
+    /// </summary>
+    /// <remarks>
+    ///     Pumps unconditionally at the top of every iteration and blocks the
+    ///     idle part of each iteration in
+    ///     <see cref="StaWaitHelper.WaitForSignalOrMessages"/> — the same pump
+    ///     discipline as <see cref="TryWaitForUpdate"/>, for the same reason.
+    /// </remarks>
+    /// <param name="serverHandle">MXAccess server handle shared by every item.</param>
+    /// <param name="itemHandles">Item handles to wait on.</param>
+    /// <param name="sinceVersions">Per-handle version baselines captured before advising.</param>
+    /// <param name="deadlineUtc">Absolute UTC deadline shared by every handle.</param>
+    /// <param name="pumpStep">Action that pumps any pending Windows messages.</param>
+    /// <param name="received">Output: <see langword="true"/> at index i when handle i received an update before the deadline.</param>
+    /// <param name="values">Output: the captured value at index i when <paramref name="received"/>[i] is set.</param>
+    /// <param name="pollIntervalMs">Upper bound on one idle wait slice.</param>
+    /// <returns>The number of handles that received an update before the deadline.</returns>
+    public int TryWaitForUpdates(
+        int serverHandle,
+        IReadOnlyList<int> itemHandles,
+        IReadOnlyList<ulong> sinceVersions,
+        DateTime deadlineUtc,
+        Action pumpStep,
+        bool[] received,
+        CachedValue[] values,
+        int pollIntervalMs = 5)
+    {
+        if (itemHandles is null)
+        {
+            throw new ArgumentNullException(nameof(itemHandles));
+        }
+
+        if (sinceVersions is null)
+        {
+            throw new ArgumentNullException(nameof(sinceVersions));
+        }
+
+        if (pumpStep is null)
+        {
+            throw new ArgumentNullException(nameof(pumpStep));
+        }
+
+        if (received is null)
+        {
+            throw new ArgumentNullException(nameof(received));
+        }
+
+        if (values is null)
+        {
+            throw new ArgumentNullException(nameof(values));
+        }
+
+        int count = itemHandles.Count;
+        if (sinceVersions.Count != count || received.Length < count || values.Length < count)
+        {
+            throw new ArgumentException("Handle, baseline, and output arrays must have matching lengths.");
+        }
+
+        int receivedCount = 0;
+        for (int index = 0; index < count; index++)
+        {
+            if (received[index])
+            {
+                receivedCount++;
+            }
+        }
+
+        while (true)
+        {
+            pumpStep();
+
+            for (int index = 0; index < count; index++)
+            {
+                if (received[index])
+                {
+                    continue;
+                }
+
+                if (TryGet(serverHandle, itemHandles[index], out CachedValue value) && value.Version > sinceVersions[index])
+                {
+                    received[index] = true;
+                    values[index] = value;
+                    receivedCount++;
+                }
+            }
+
+            if (receivedCount >= count)
+            {
+                return receivedCount;
+            }
+
+            TimeSpan remaining = deadlineUtc - DateTime.UtcNow;
+            if (remaining <= TimeSpan.Zero)
+            {
+                return receivedCount;
+            }
+
+            StaWaitHelper.WaitForSignalOrMessages(
+                updateSignal,
+                StaWaitHelper.ClampWaitMilliseconds(remaining, pollIntervalMs));
+        }
+    }
+
     /// <summary>Returns the current version for a handle pair, or 0 if no entry exists.</summary>
     /// <param name="serverHandle">MXAccess server handle.</param>
     /// <param name="itemHandle">MXAccess item handle.</param>

src/ZB.MOM.WW.MxGateway.Worker/MxAccess/MxAccessWriteCompletionCache.cs

@@ -184,6 +184,57 @@ public sealed class MxAccessWriteCompletionCache
         }
     }
 
+    /// <summary>
+    ///     Forgets the completion entry of a handle pair whose MXAccess item was
+    ///     removed, so a long-lived writer that drops and re-adds items does not
+    ///     keep one entry per dropped handle for the life of the worker.
+    /// </summary>
+    /// <param name="serverHandle">MXAccess server handle.</param>
+    /// <param name="itemHandle">MXAccess item handle that was removed.</param>
+    public void Remove(
+        int serverHandle,
+        int itemHandle)
+    {
+        lock (syncRoot)
+        {
+            entries.Remove(CreateItemKey(serverHandle, itemHandle));
+        }
+    }
+
+    /// <summary>Forgets every completion entry recorded under a server handle that was unregistered.</summary>
+    /// <param name="serverHandle">MXAccess server handle that was unregistered.</param>
+    public void RemoveServer(int serverHandle)
+    {
+        lock (syncRoot)
+        {
+            List<long> stale = new();
+            foreach (long key in entries.Keys)
+            {
+                if ((int)(key >> 32) == serverHandle)
+                {
+                    stale.Add(key);
+                }
+            }
+
+            foreach (long key in stale)
+            {
+                entries.Remove(key);
+            }
+        }
+    }
+
+    /// <summary>Gets the number of handle pairs with a recorded completion (test seam).</summary>
+    internal int Count
+    {
+        get
+        {
+            lock (syncRoot)
+            {
+                return entries.Count;
+            }
+        }
+    }
+
     private static long CreateItemKey(
         int serverHandle,
         int itemHandle)

src/ZB.MOM.WW.MxGateway.Worker/MxAccess/WnWrapAlarmConsumer.cs

@@ -202,14 +202,14 @@ public sealed class WnWrapAlarmConsumer : IMxAccessAlarmConsumer
 
             // Use the IwwAlarmConsumer (v1) prefix-named methods for the
             // lifecycle. Empirically (live dev-rig 2026-05-01, reconfirmed
-            // 2026-08-18) this is the only path that lets AlarmAckByName
+            // 2026-08-18) the v1 path is the one that leaves the consumer in
-            // return rc=0 afterwards. The v2 Initialize/Register/Subscribe
+            // a usable state: the v2 Initialize/Register/Subscribe methods on
-            // methods on the class succeed (return 0) but acks against that
+            // the class succeed (return 0) but later calls against that
             // consumer state return -55. The v1 prefix path is what
-            // WIN-911-style code uses against the same wnwrap library.
+            // WIN-911-style code uses against the same wnwrap library. This
-            // Note rc=0 means the call was
+            // consumer only reads the alarm snapshot; acknowledgements go
-            // accepted, not that an acknowledgement was applied — see
+            // through AlarmAckWriter, not through wnwrap (see
-            // docs/AlarmProbeFindings.md. Acks now go through AlarmAckWriter.
+            // docs/AlarmProbeFindings.md).
             int init = com.IwwAlarmConsumer_InitializeConsumer(DefaultApplicationName);
             if (init != 0)
             {
@@ -615,10 +615,10 @@ public sealed class WnWrapAlarmConsumer : IMxAccessAlarmConsumer
     ///         place under its original GUID and the re-raise replaced it with a
     ///         freshly minted one. That is the intended reading — the old
     ///         instance ended and a new one began — so the diff emits a Clear
-    ///         plus a Raise, which is what happened. The acknowledge leg remains
+    ///         plus a Raise, which is what happened. Every alarm-client ack surface
-    ///         assumed rather than observed: on the dev rig every ack surface is
+    ///         is inert on the dev rig (it returns 0 and nothing moves), so
-    ///         inert (<c>AlarmAckByName</c> returns 0 and nothing moves), so no
+    ///         acknowledgements go through the AckMsg writer; an AckMsg write
-    ///         record can be driven into an acknowledged state — see
+    ///         moves the record to <c>ACK_ALM</c> (observed 2026-10-19) — see
     ///         <c>docs/AlarmProbeFindings.md</c>.
     ///     </para>
     /// </remarks>

src/ZB.MOM.WW.MxGateway.Worker/Sta/IStaMessagePump.cs

@@ -0,0 +1,22 @@
+using System;
+using System.Threading;
+
+namespace ZB.MOM.WW.MxGateway.Worker.Sta;
+
+/// <summary>
+///     The two Win32 pump operations <see cref="StaRuntime"/> drives on its
+///     STA thread. <see cref="StaMessagePump"/> is the only production
+///     implementation; the seam exists so Worker.Tests can observe pump
+///     interleaving and inject a pump failure without a real message queue.
+/// </summary>
+internal interface IStaMessagePump
+{
+    /// <summary>Waits for a command wake event or Windows messages.</summary>
+    /// <param name="commandWakeEvent">Event signalled when command work is available.</param>
+    /// <param name="timeout">Maximum time to wait.</param>
+    void WaitForWorkOrMessages(WaitHandle commandWakeEvent, TimeSpan timeout);
+
+    /// <summary>Pumps and dispatches all pending Windows messages.</summary>
+    /// <returns>The number of messages dispatched.</returns>
+    int PumpPendingMessages();
+}

src/ZB.MOM.WW.MxGateway.Worker/Sta/StaMessagePump.cs

@@ -6,7 +6,7 @@ using Microsoft.Win32.SafeHandles;
 namespace ZB.MOM.WW.MxGateway.Worker.Sta;
 
 /// <summary>Pumps Windows messages on the STA thread to allow MXAccess COM events to deliver.</summary>
-public sealed class StaMessagePump
+public sealed class StaMessagePump : IStaMessagePump
 {
     private const uint Infinite = 0xFFFFFFFF;
     private const uint PmRemove = 0x0001;

src/ZB.MOM.WW.MxGateway.Worker/Sta/StaRuntime.cs

@@ -8,7 +8,7 @@ namespace ZB.MOM.WW.MxGateway.Worker.Sta;
 public sealed class StaRuntime : IDisposable
 {
     private readonly IStaComApartmentInitializer comApartmentInitializer;
-    private readonly StaMessagePump messagePump;
+    private readonly IStaMessagePump messagePump;
     private readonly ConcurrentQueue<IStaWorkItem> commandQueue = new();
     private readonly AutoResetEvent commandWakeEvent = new(false);
     private readonly ManualResetEventSlim startedEvent = new(false);
@@ -20,6 +20,8 @@ public sealed class StaRuntime : IDisposable
     private bool startRequested;
     private bool shutdownRequested;
     private Exception? startupException;
+    private Exception? loopFault;
+    private Action<Exception>? loopFaultHandler;
     private long lastActivityUtcTicks;
     private bool comInitialized;
 
@@ -41,6 +43,21 @@ public sealed class StaRuntime : IDisposable
         IStaComApartmentInitializer comApartmentInitializer,
         StaMessagePump messagePump,
         TimeSpan idlePumpInterval)
+        : this(comApartmentInitializer, (IStaMessagePump)messagePump, idlePumpInterval)
+    {
+    }
+
+    /// <summary>
+    /// Initializes a new instance of <see cref="StaRuntime"/> over any
+    /// <see cref="IStaMessagePump"/> (test seam).
+    /// </summary>
+    /// <param name="comApartmentInitializer">COM apartment initializer.</param>
+    /// <param name="messagePump">Message pump for the STA thread.</param>
+    /// <param name="idlePumpInterval">Interval for idle message pump waits.</param>
+    internal StaRuntime(
+        IStaComApartmentInitializer comApartmentInitializer,
+        IStaMessagePump messagePump,
+        TimeSpan idlePumpInterval)
     {
         this.comApartmentInitializer = comApartmentInitializer
             ?? throw new ArgumentNullException(nameof(comApartmentInitializer));
@@ -79,6 +96,59 @@ public sealed class StaRuntime : IDisposable
     /// </summary>
     public bool IsRunning => startedEvent.IsSet && !stoppedEvent.IsSet;
 
+    /// <summary>
+    /// Gets the exception that terminated the STA loop after a successful
+    /// <see cref="Start"/>, or <see langword="null"/> while the loop is healthy
+    /// (or stopped by a requested shutdown). Startup failures are not reported
+    /// here — <see cref="Start"/> throws them.
+    /// </summary>
+    public Exception? LoopFault
+    {
+        get
+        {
+            lock (gate)
+            {
+                return loopFault;
+            }
+        }
+    }
+
+    /// <summary>
+    /// Registers the callback invoked once, on the dying STA thread, when the
+    /// STA loop terminates with an exception after a successful
+    /// <see cref="Start"/>. When the loop has already died the callback runs
+    /// immediately on the calling thread, so a fault that lands between
+    /// <see cref="Start"/> and registration is never lost. Only one handler is
+    /// kept; a later registration replaces an earlier one.
+    /// </summary>
+    /// <remarks>
+    /// Without this, a post-start loop death (a failed
+    /// <c>MsgWaitForMultipleObjectsEx</c>, or an exception escaping a dispatched
+    /// window message) left the worker process alive with no STA: every later
+    /// command queued forever and the gateway only learned of it through the
+    /// heartbeat watchdog, if at all.
+    /// </remarks>
+    /// <param name="handler">Callback receiving the terminating exception.</param>
+    public void RegisterLoopFaultHandler(Action<Exception> handler)
+    {
+        if (handler is null)
+        {
+            throw new ArgumentNullException(nameof(handler));
+        }
+
+        Exception? alreadyFaulted;
+        lock (gate)
+        {
+            loopFaultHandler = handler;
+            alreadyFaulted = shutdownRequested ? null : loopFault;
+        }
+
+        if (alreadyFaulted is not null)
+        {
+            handler(alreadyFaulted);
+        }
+    }
+
     /// <summary>
     ///     Pumps any pending Windows messages on the calling thread and refreshes
     ///     the STA activity timestamp. Intended for commands that synchronously
@@ -176,6 +246,14 @@ public sealed class StaRuntime : IDisposable
 
         lock (gate)
         {
+            if (loopFault is not null)
+            {
+                // The loop is gone; nothing would ever dequeue this item.
+                return Task.FromException<T>(new InvalidOperationException(
+                    "The worker STA loop terminated unexpectedly.",
+                    loopFault));
+            }
+
             if (shutdownRequested)
             {
                 return Task.FromException<T>(new StaRuntimeShutdownException());
@@ -244,12 +322,14 @@ public sealed class StaRuntime : IDisposable
 
     private void ThreadMain()
     {
+        bool started = false;
         try
         {
             StaThreadId = Thread.CurrentThread.ManagedThreadId;
             comApartmentInitializer.Initialize();
             comInitialized = true;
             MarkActivity();
+            started = true;
             startedEvent.Set();
 
             while (!IsShutdownRequested())
@@ -264,8 +344,44 @@ public sealed class StaRuntime : IDisposable
         }
         catch (Exception exception)
         {
-            startupException = exception;
+            if (!started)
-            startedEvent.Set();
+            {
+                // Start() is blocked on startedEvent and rethrows this.
+                startupException = exception;
+                startedEvent.Set();
+            }
+            else
+            {
+                // Post-start loop death. Start() returned long ago, so nothing
+                // would ever read startupException: record the fault where
+                // InvokeAsync and the registered handler see it. Set under the
+                // gate so every item enqueued before it is still in the queue
+                // for the finally block's CancelQueuedCommands, and every
+                // InvokeAsync after it fails fast instead of queueing forever.
+                Action<Exception>? handler;
+                lock (gate)
+                {
+                    loopFault = exception;
+
+                    // A loop that throws while a requested shutdown is already
+                    // in progress is being torn down on purpose; reporting it
+                    // as a worker fault would turn a clean stop into a fault.
+                    handler = shutdownRequested ? null : loopFaultHandler;
+                }
+
+                if (handler is not null)
+                {
+                    try
+                    {
+                        handler(exception);
+                    }
+                    catch
+                    {
+                        // The handler reports the fault; a failure inside it
+                        // must not skip the apartment cleanup below.
+                    }
+                }
+            }
         }
         finally
         {
@@ -292,6 +408,17 @@ public sealed class StaRuntime : IDisposable
             MarkActivity();
             workItem.Execute();
             MarkActivity();
+
+            // Pump between work items, not only once the whole queue is
+            // drained: a burst of queued commands (bulk callers, the alarm
+            // poll, a reconnecting client replaying subscriptions) would
+            // otherwise hold every inbound COM callback — OnDataChange,
+            // OnWriteComplete — behind the entire backlog. PeekMessage with an
+            // empty queue is cheap, so the per-item cost is negligible.
+            if (!commandQueue.IsEmpty)
+            {
+                messagePump.PumpPendingMessages();
+            }
         }
     }
 

src/ZB.MOM.WW.MxGateway.Worker/WorkerApplication.cs

@@ -14,10 +14,61 @@ public static class WorkerApplication
     /// <returns>The process exit code.</returns>
     public static int Run(string[] args)
     {
+        WorkerConsoleLogger logger = new(Console.Error);
+
+        // WRK-09: installed only on the production entry point (not the
+        // injectable overloads tests call) because the hook is process-global.
+        AppDomain.CurrentDomain.UnhandledException += (_, eventArgs) =>
+            HandleUnhandledException(
+                logger,
+                eventArgs.ExceptionObject,
+                eventArgs.IsTerminating,
+                Environment.Exit);
+
         return Run(
             args,
             new EnvironmentVariableWorkerEnvironment(),
-            new WorkerConsoleLogger(Console.Error));
+            logger);
+    }
+
+    /// <summary>
+    ///     Body of the worker's <c>AppDomain.CurrentDomain.UnhandledException</c>
+    ///     hook. Logs one <c>WorkerUnhandledException</c> line and exits the
+    ///     process with <see cref="WorkerExitCode.UnhandledException"/>, so the
+    ///     gateway's launcher sees a defined worker exit code rather than the
+    ///     CLR's crash code.
+    /// </summary>
+    /// <remarks>
+    ///     Logs the exception type only — never its message or stack, which
+    ///     can carry tag values or credentials passed through a command — the
+    ///     same rule every other worker failure line follows. A logging
+    ///     failure is swallowed so the exit always happens.
+    /// </remarks>
+    /// <param name="logger">Worker logger.</param>
+    /// <param name="exceptionObject">The unhandled exception object (not necessarily an <see cref="Exception"/>).</param>
+    /// <param name="isTerminating">Whether the CLR is terminating the process.</param>
+    /// <param name="exit">Process-exit action; <see cref="Environment.Exit"/> in production.</param>
+    internal static void HandleUnhandledException(
+        IWorkerLogger logger,
+        object? exceptionObject,
+        bool isTerminating,
+        Action<int> exit)
+    {
+        try
+        {
+            logger.Error("WorkerUnhandledException", new Dictionary<string, object?>
+            {
+                ["exit_code"] = WorkerExitCode.UnhandledException,
+                ["exception_type"] = exceptionObject?.GetType().FullName ?? "null",
+                ["is_terminating"] = isTerminating,
+            });
+        }
+        catch
+        {
+            // Never let a logging failure skip the defined exit.
+        }
+
+        exit((int)WorkerExitCode.UnhandledException);
     }
 
     /// <summary>Initializes and runs the worker with custom environment and logging.</summary>