.NET 4.5: internal error in the .NET Runtime (80131506) / disabling concurrent GC

Question

I have a long-running .NET 4.5 application that crashes randomly, leaving the message I've mentioned in the question title in the event log. The issue is reproduced on 3 different machines and 2 different systems (2008 R2 and 2012). Application doesn't use any unsafe/unmanaged components, it's pure managed .NET, with the only unmanaged thing being the CLR itself.

Here's the stack trace of the crash site that I've extracted from the dump:

clr.dll!MethodTable::GetCanonicalMethodTable()  
clr.dll!SVR::CFinalize::ScanForFinalization()  - 0x1a31b bytes  
clr.dll!SVR::gc_heap::mark_phase()  + 0x328 bytes   
clr.dll!SVR::gc_heap::gc1()  + 0x95 bytes   
clr.dll!SVR::gc_heap::garbage_collect()  + 0x16e bytes  
clr.dll!SVR::gc_heap::gc_thread_function()  + 0x3e bytes    
clr.dll!SVR::gc_heap::gc_thread_stub()  + 0x77 bytes    
kernel32.dll!BaseThreadInitThunk()  + 0x1a bytes    
ntdll.dll!RtlUserThreadStart()  + 0x21 bytes    

This issue closely resembles the one that was discussed here, so I tried the solutions suggested in that topic, but none of them helped:

• I've tried installing this hotfix, but it won't install on any of my machines (KB2640103 does not apply, or is blocked by another condition on your computer), which actually makes sense, because I'm using 4.5, not 4.0.

• I've tried disabling concurrent GC and/or enabling server GC. Right now the relevant part of my app.config looks like this:

  <?xml version="1.0"?>
  <configuration>        
      <runtime>
          <gcConcurrent enabled="false"/>
          <gcServer enabled="true" />
      </runtime>
  <startup><supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5"/>    </startup></configuration>
  

Though the weird thing is I still find multiple GC-related threads in the process dump. Besides the one the crash occurs in, there are 7 threads with the following stack trace:

ntdll.dll!NtWaitForSingleObject()  + 0xa bytes  
KERNELBASE.dll!WaitForSingleObjectEx()  + 0x9a bytes    
clr.dll!CLREventBase::WaitEx()  + 0x13f bytes   
clr.dll!CLREventBase::WaitEx()  + 0xf7 bytes    
clr.dll!CLREventBase::WaitEx()  + 0x78 bytes    
clr.dll!SVR::t_join::join()  + 0xd8 bytes   
clr.dll!SVR::gc_heap::scan_dependent_handles()  + 0x65 bytes    
clr.dll!SVR::gc_heap::mark_phase()  + 0x347 bytes   
clr.dll!SVR::gc_heap::gc1()  + 0x95 bytes   
clr.dll!SVR::gc_heap::garbage_collect()  + 0x16e bytes  
clr.dll!SVR::gc_heap::gc_thread_function()  + 0x3e bytes    
clr.dll!SVR::gc_heap::gc_thread_stub()  + 0x77 bytes    
kernel32.dll!BaseThreadInitThunk()  + 0x1a bytes    
ntdll.dll!RtlUserThreadStart()  + 0x21 bytes    

Which makes me wondering if I could somehow screw up disabling the concurrent GC (that's what I actually listed the config for).

I think that wraps up what I've managed to find so far. I could really use some help on how to proceed with dealing with this issue.

Answer 1

I am drawing from my past experience in our application. This could be caused if an exception goes unhandled till the Finalizer level, and if it goes... it will crash the application.

Before doing anything on the GC configuration..

One quick check... Are you using task parallel libraries?. If yes make sure you are handling exceptions properly. If exceptions from different threads are left unhandled it goes till Finalizer which then crashes the application. There are couple of ways to handle them neatly. Handling 'Aggregate' Exception is one way (that we used to solve!).

http://msdn.microsoft.com/en-us/library/dd537614.aspx

I don't have 50 points to add a comment, so adding it as an answer...

Answer 2

I realize this is an old post, however, I ran into the same issue as the OP. The point atlaste made:

Change the runtime to x86 or x64 and try again; you can also mess with the concurrent GC settings like you already tried.

Was the key for me. All of my projects were set to Any CPU except for one (coincidentally the entry point for the application which is a Console Application project). This project was set to x86. Once I changed it to Any CPU the application ran correctly.

Answer 3

Solution that helped me: uninstall .NET 4.5.1, install 4.0, install mentioned hotfix, install 4.5.1 back.

Answer 4

I just finished a conversation with Microsoft since I have been able to reproduce an issue which is similar.

In my case it was a bug in the .NET runtime, which has to do with mixing dynamic types and non-dynamic code. I'm not sure if this is also the case in your scenario, but there are some thing you might want to try:

• Run the code on Windows 8.1 (latest updates). Apparently Windows 8.1 has a more recent version of .NET than the other versions of Windows.
• If you use AssemblyBuilder (like I did), try to change it to Run mode instead of RunAndCollect.
• Change the runtime to x86 or x64 and try again; you can also mess with the concurrent GC settings like you already tried.
• My bug is being fixed as we speak, which basically means there'll be a windows update that took care of it. Perhaps it's also an option to simply wait for that; I don't expect that to take too long, since it's quite critical for a lot of programs.

Answer 5

I finally found a fix I could install. I also have 4.5 and other fix for 4.0 was not being installed. Removing 4.5 did not fix it either. Fix in the link actually fixed it.

http://kb.machsol.com/Knowledgebase/Article/50305

Answer 6

We've had the same problem in our .NET 4.5 desktop app - web scraper. It crashed randomly under heavy load. So we've been searching for ways to find out what was the cause for a few months: we've tried everything! Disabling concurrent GC, setting it to Server mode and many-many other workarounds, until we've realized that the crashes occured because of the PhantomJS module. It uses some unmanaged resources and doesn't clear them afterwards :( So we've created a stand-alone console app for PhantomJS integration. Now we execute this console app with Process.Start from out web scraper and kill it afterwards. It takes more time for scraping, but no more crashes!

Answer 7

My problem is weird byt every 5-10 minutes my application pool kept crashing with this exit code (80131506). I'm not sure in high threaded operation / schedule task you should thrust the Garbage Collector, but the following solution worked for here.

I added a Job that calls GC.GetTotalMemory(true) every minute. I assume that, for some reason, the GC is not automatically calling the Garbage Collector often enough for the high number of disposable objects that I use. But this fix my problem! It's more like a quick fix than a final solution ;)