scadalink-design/deprecated/lmxproxy/docs/sta_gap.md

STA Message Pump Gap — OnWriteComplete COM Callback

Status: Documented gap. Fire-and-forget workaround in place (deviation #7). Full fix deferred until secured/verified writes are needed.

When This Matters

The current fire-and-forget write approach works for supervisory writes where:

• Security is handled at the LmxProxy API key level, not MxAccess attribute level
• Writes succeed synchronously (no secured/verified write requirements)
• Write confirmation is handled at the application level (read-back in WriteBatchAndWait)

This gap becomes a blocking issue if any of these scenarios arise:

• Secured writes (MxAccess error 1012): Attribute requires ArchestrA user authentication. OnWriteComplete returns the error, and the caller must retry with WriteSecured().
• Verified writes (MxAccess error 1013): Attribute requires two-user verification. Same retry pattern.
• Write failure detection: MxAccess accepts the Write() call but can't complete it (e.g., downstream device failure). OnWriteComplete is the only notification of this — without it, the caller assumes success.

Root Cause

The MxAccess documentation (Write() Method) states: "Upon completion of the write, your program receives notification of the success/failure status through the OnWriteComplete() event" and "that item should not be taken off advise or removed from the internal tables until the OnWriteComplete() event is received."

OnWriteComplete should fire after every Write() call. It doesn't in our service because:

• MxAccess is a COM component designed for Windows Forms apps with a UI message loop
• COM event callbacks are delivered via the Windows message pump
• Our Topshelf Windows service has no message pump — Write() is called from thread pool threads (Task.Run) with no message loop
• OnDataChange works because MxAccess fires it proactively on its own internal threads; OnWriteComplete is a response callback that needs message-pump-based marshaling

Correct Solution: Dedicated STA Thread + Application.Run()

Based on research (Stephen Toub, MSDN Magazine 2007; Microsoft Learn COM interop docs; community patterns), the correct approach is a dedicated STA thread running a Windows Forms message pump via Application.Run().

Architecture

Service main thread (MTA)
    │
    ├── gRPC server threads (handle client RPCs)
    │       │
    │       └── Marshal COM calls via Form.BeginInvoke() ──┐
    │                                                       │
    └── Dedicated STA thread                                │
            │                                               │
            ├── Creates LMXProxyServerClass COM object      │
            ├── Wires event handlers (OnDataChange,         │
            │   OnWriteComplete, OperationComplete)         │
            ├── Runs Application.Run() ← continuous         │
            │   message pump                                │
            │                                               │
            └── Hidden Form receives BeginInvoke calls ◄────┘
                    │
                    ├── Executes COM operations (Read, Write,
                    │   AddItem, AdviseSupervisory, etc.)
                    │
                    └── COM callbacks delivered via message pump
                        (OnWriteComplete, OnDataChange, etc.)

Implementation Pattern

// In MxAccessClient constructor or Start():
var initDone = new ManualResetEventSlim(false);

_staThread = new Thread(() =>
{
    // 1. Create hidden form for marshaling
    _marshalForm = new Form();
    _marshalForm.CreateHandle(); // force HWND creation without showing

    // 2. Create COM objects ON THIS THREAD
    _lmxProxy = new LMXProxyServerClass();
    _lmxProxy.OnDataChange += OnDataChange;
    _lmxProxy.OnWriteComplete += OnWriteComplete;

    // 3. Signal that init is complete
    initDone.Set();

    // 4. Run message pump (blocks forever, pumps COM callbacks)
    Application.Run();
});
_staThread.Name = "MxAccess-STA";
_staThread.IsBackground = true;
_staThread.SetApartmentState(ApartmentState.STA);
_staThread.Start();

initDone.Wait(); // wait for COM objects to be ready

Dispatching Work to the STA Thread

// All COM calls must go through the hidden form's invoke:
public Task<Vtq> ReadAsync(string address, CancellationToken ct)
{
    var tcs = new TaskCompletionSource<Vtq>();
    _marshalForm.BeginInvoke((Action)(() =>
    {
        try
        {
            // COM call executes on STA thread
            int handle = _lmxProxy.AddItem(_connectionHandle, address);
            _lmxProxy.AdviseSupervisory(_connectionHandle, handle);
            // ... etc
            tcs.SetResult(vtq);
        }
        catch (Exception ex)
        {
            tcs.SetException(ex);
        }
    }));
    return tcs.Task;
}

Shutdown

// To stop the message pump:
_marshalForm.BeginInvoke((Action)(() =>
{
    // Clean up COM objects on STA thread
    // ... UnAdvise, RemoveItem, Unregister ...
    Marshal.ReleaseComObject(_lmxProxy);
    Application.ExitThread(); // stops Application.Run()
}));
_staThread.Join(TimeSpan.FromSeconds(10));

Why Our First Attempt Failed

Our original StaDispatchThread (Phase 2) used BlockingCollection.Take() to wait for work items, with Application.DoEvents() between items. This failed because:

Our failed approach	Correct approach
BlockingCollection.Take() blocks the STA thread, preventing the message pump from running	Application.Run() runs continuously, pumping messages at all times
Application.DoEvents() only pumps messages already in the queue at that instant	Message pump runs an infinite loop, processing messages as they arrive
Work dispatched by enqueueing to BlockingCollection	Work dispatched via Form.BeginInvoke() which posts a Windows message to the STA thread's queue

The key difference: BeginInvoke posts a WM_ message that the message pump processes alongside COM callbacks. BlockingCollection bypasses the message pump entirely.

Drawbacks of the STA Approach

Performance

• All COM calls serialize onto one thread. Under load (batch reads of 100+ tags), operations queue up single-file. Current Task.Run approach allows MxAccess's internal marshaling to handle some concurrency.
• Double context switch per operation. Caller → STA thread (invoke) → wait → back to caller. Adds ~0.1-1ms per call. Negligible for single reads, noticeable for large batch operations.

Safety

• Single point of failure. If the STA thread dies, all MxAccess operations stop. Recovery requires tearing down and recreating the thread + all COM objects.
• Deadlock risk. If STA thread code synchronously waits on something that needs the STA thread (circular dependency), the message pump freezes. All waits must be async/non-blocking.
• Reentrancy. While pumping messages, inbound COM callbacks can reenter your code during another COM call. Event handlers must be reentrant-safe.

Complexity

• Every COM call needs _marshalForm.BeginInvoke() wrapping.
• COM object affinity to STA thread is hard to enforce at compile time.
• Unit tests need STA thread support or must use fakes.

Decision

Fire-and-forget is the correct choice for now. Revisit when secured/verified writes are needed.

References

• .NET Matters: Handling Messages in Console Apps — Stephen Toub, MSDN Magazine 2007
• How to: Support COM Interop by Displaying Each Windows Form on Its Own Thread — Microsoft Learn
• .NET Windows Service needs STAThread — hirenppatel
• Application.Run() In a Windows Service — PC Review
• Build a message pump for a Windows service? — CodeProject
• MxAccess Toolkit User's Guide — Write() Method, OnWriteComplete Callback sections