perf(worker,gateway): pin MXSTATUS_PROXY record info, dedicated event-drain thread, cap thread-pool spin
At ~240 events/s the worker burned 8-12% system CPU on wonder-app-vd03. A WPR trace put 72% of it in lmxproxy's per-event GetRecordInfoFromGuids (type library reload through the registry, amplified by WOW64 and the EDR registry filter) and 26% in the .NET Framework thread pool spinning after each per-event work item. - Worker: MxStatusProxyRecordInfoPin holds the LMXProxy ITypeLib and the MXSTATUS_PROXY IRecordInfo for the process lifetime, so oleaut32 serves the per-event lookup from cache (~1 ms -> <1 us per call). Best-effort, never fatal; status logged at handshake. - Worker: the event drain loop runs on a dedicated background thread with a synchronous wait and a synchronous pipe flush, removing the per-event thread-pool hops. - Gateway: runtimeconfig sets ThreadPool.UnfairSemaphoreSpinLimit=10 to cut PortableThreadPool spin-wait CPU. windev bench at ~242 ev/s: worker 67% -> 6.2% of one core; gateway 8.1% -> 6.9%. Worker suite 785 passed / 14 skipped (x86, windev).
This commit is contained in:
@@ -479,6 +479,32 @@ Creation rules:
|
||||
- Never marshal the raw COM object to pipe reader/writer threads.
|
||||
- Capture COM creation HRESULT or exception details.
|
||||
|
||||
**Record-info pin.** Before the first `LMXProxyServerClass` is created,
|
||||
`MxAccessComObjectFactory.Create()` calls `MxStatusProxyRecordInfoPin.EnsurePinned()`. Once per
|
||||
process, this loads the LMXProxy type library and the `MXSTATUS_PROXY` record info, and holds
|
||||
both references until exit. The GUIDs and version come from the interop assembly's metadata:
|
||||
library `{77e896ec-b3e3-4de4-b96f-f32570164211}` v3.2 (`MXAccess32.tlb`), record
|
||||
`{A162D9E4-9019-42F3-A087-79264007E542}`.
|
||||
|
||||
The pin exists because every `OnDataChange` carries a SAFEARRAY of `MXSTATUS_PROXY` records, and
|
||||
`lmxproxy.dll` resolves that record type with `GetRecordInfoFromGuids` each time it builds one.
|
||||
With no live reference, oleaut32 re-reads the registry and reloads the type library on every
|
||||
call:
|
||||
|
||||
- about 1 ms per event on the dev box;
|
||||
- worse under WOW64 registry redirection plus an EDR registry filter, where a production WPR trace
|
||||
put it at ~70% of the STA's CPU.
|
||||
|
||||
While one `IRecordInfo` is held, the same call returns oleaut32's cached instance in under a
|
||||
microsecond. Measured at ~240 events/s on the dev box, worker CPU fell from 67% to 10% of a core
|
||||
with the pin alone.
|
||||
|
||||
The pin only warms a cache. It changes no MXAccess behaviour, and a failure is never fatal. The
|
||||
outcome is in `MxStatusProxyRecordInfoPin.Status`, and the worker logs it as `WorkerRecordInfoPin`
|
||||
just before `WorkerReady`. Worker logs go to stderr, which the gateway does not redirect, so the
|
||||
line lands wherever the host captures the gateway's stderr (the NSSM stderr log on deployed
|
||||
boxes).
|
||||
|
||||
If COM creation fails, the worker should send a structured fault with:
|
||||
|
||||
- fault category,
|
||||
@@ -564,9 +590,27 @@ everything that arrived — the loop must therefore re-check `DrainFault()` and
|
||||
re-drain after every wait, never treat a wake as "exactly one event". The 25 ms
|
||||
`EventDrainInterval` survives as the **fallback ceiling** on an unsignalled wait,
|
||||
not as a latency floor: an event arriving at an idle worker is framed at signal
|
||||
latency instead of waiting out a tick, an idle worker parks instead of waking 40
|
||||
times a second, and the interval only bounds how long the loop may sleep if some
|
||||
future path mutates the queue without signalling.
|
||||
latency instead of waiting out a tick, and the interval only bounds how long the
|
||||
loop may sleep if some future path mutates the queue without signalling. An idle
|
||||
worker still wakes at that ceiling, but a wake on an empty queue is cheap.
|
||||
|
||||
The loop runs on a **dedicated background thread** (`mxgw-event-drain`), not on the
|
||||
CLR thread pool:
|
||||
|
||||
- It parks in the blocking `IWorkerRuntimeSession.WaitForEvents`, which uses the
|
||||
same coalesced signal and single-waiter contract as `WaitForEventsAsync`.
|
||||
- It blocks on the frame writer's batch write.
|
||||
- `RunEventDrainLoopAsync` starts the thread and returns a task that completes,
|
||||
faults or cancels exactly as the loop does, so the message loop's `Task.WhenAny`
|
||||
supervision is unchanged.
|
||||
|
||||
The reason is .NET Framework's thread pool. As an async loop, every enqueue's
|
||||
semaphore release queued a continuation to the pool, and each pool worker that ran
|
||||
one then spun in `SleepEx` before parking. A production WPR trace measured ~150
|
||||
context switches per event and ~26% of the worker's CPU in that spin. With the
|
||||
dedicated thread and the frame writer's synchronous flush, the steady-state event
|
||||
path schedules no managed pool work. The overlapped pipe write still completes on
|
||||
an I/O completion thread.
|
||||
|
||||
A single event whose envelope exceeds the negotiated frame maximum is
|
||||
**undeliverable end to end** — the pipe maximum sits only the envelope-overhead
|
||||
|
||||
Reference in New Issue
Block a user