Memory Overflow: Having an increasing number of Microsoft.CSharp.RuntimeBinder.Semantics

Question

We are currently hunting some memory leaks in our application, when doing some operation(loading and closing one project inside our application), we know that the memory inc

Answer 1

I encountered exactly the same issue today by profiling memory leaks in my app RepoZ. That tool is supposed to run in the background, checking Git repositories and update Windows Explorer window titles periodically. The latter task has to make some COM calls to "Shell.Application" to find the Explorer windows and determine the path they are currently pointing to.

By using the dynamic keyword like this ...

dynamic shell = Activator.CreateInstance(...);
foreach (object window in shell.Windows())
{ 
    var hwnd = window.Hwnd;
    ...
}

... I ended up to a memory dump like that after a few hours:

Combridge

To solve that, I wrote a little helper class called "Combridge" caring to release COM objects and providing quite easy access to methods and properties of the underlying COM object. It is very easy and straightforward, nothing special here. It makes use of Reflection to COM objects, that's why there's some loss in performance (see below).

With it, the code sample from above looks like this:

using (var shell = new Combridge(Activator.CreateInstance(...)))
{
    var windows = shell.InvokeMethod<IEnumerable>("Windows");
    foreach (var window in windows)
    {
        var hwnd = window.GetPropertyValue<long>("Hwnd");
        ... 
    }
}

You can see the file ExplorerWindowActor on how it is used in RepoZ.

It is not exactly as beautiful as with dynamic and performance got worse in this first attempt as well. A quick bench showed the following:

Performance

I tested 1000 iterations, in each iteration 10 open Explorer windows were processed. For each window, 4 methods or properties are invoked on that COM object. So we're talking about 40.000 COM calls.

The duration went up from ~2500ms (dynamic) to ~6000ms (Combridge). That's from 0.062ms to 0.150ms for each call.

So this takes about 2.4x the time to finish now.

This is significant, I know. But it is okay for my requirements and the memory leak is gone.

That's it - I wanted to share that story with you, hopefully you can use that class (or an improved version of it) to get out of the dynamic hell as well.

~Update~

After 10 hours, RepoZ still runs with a very constant memory footprint.

So with 10 Explorer windows open, 4 COM calls per window and that whole loop two times a second, RepoZ created about 72.000 COM instances and made about 2.880.000 COM calls in total without any increase in memory consumption.

I guess we can say that the issue really comes with dynamic.

Answer 2

Dynamic keyword should be used seldom since in most of the cases workarounds can be found not requiring it.

Based on your application, the best advise is to carefully think if you can design your solution so that you avoid dynamic. Here are some valid use cases for dynamic: https://msdn.microsoft.com/en-us/library/dd264736.aspx

Given that you really need to use dynamic, I would suggest instrumenting your code and figure out what parts are the most memory consuming. Indeed using dynamic increases your memory consumption based on the fact that it needs to perform all kinds of lookups, but to have a out-of-memory exception you would need to use a lot of dynamic variables for a lot of unknown types.

There are a lot of different ways for calling methods on unknown types, and measuring and tuning the bottlenecks is the way to go.

PS: Also, posting some code snippets helps a lot.