Intermittent RPC_E_DISCONNECTED from in-process cross apartment calls

Question

I am struggling with COM functions returning RPC_E_DISCONNECTED intermittently when performing cross apartment function calls from a client to a COM server. Most of the time