Memory Leak in StringBuilder when using Insert() and Clear()

Question

I need to add some lines to a StringBuilder, where the line added last should be at the beginning of the string and the oldest at the end. I add a new line like this:

stringBuilder.Insert(0, "Some text." + Environment.NewLine);

Once done, I empty the StringBuilder like this:

stringBuilder.Clear();

I reuse the same StringBuilder many times, which leads to an Out of Memory exception over time.

I investigated the problem with the test program below, which shows that when using Append() and Clear(), the Capacity of the StringBuilder stays the same, regardless how many times it is used. But when using Insert() and Clear(), the Capacity of the StringBuilder keeps growing. Here are the test results:

           stringbuilder.Capacity
Iteration  Append Insert
0:         81984,  78016
1:         81984, 155938
2:         81984, 233860
3:         81984, 311782
4:         81984, 389704
5:         81984, 467626
6:         81984, 545548
7:         81984, 623470
8:         81984, 701392

These test figures are from the test program below. As one can see, regardless how many times one does Append() and Clear(), the capacity does not change. But it does keep growing when using Insert() and Clear().

Question: Is it correct to say that StringBuilder has a memory leak when using Insert() and Clear(), because the Capacity keeps growing and this finally leads to an Out Of Memory exception ?

To test it for yourself, just increase the upper limit for outerIndex to get the exception.

If this should indeed be considered a memory leak, how do I report it to Microsoft ?

using System;
using System.Text;

namespace TestStringBuilder {
  class Program {
    static void Main(string[] args) {
      StringBuilder stringBuilderAppend = new StringBuilder();
      StringBuilder stringBuilderInsert = new StringBuilder();
      string capacityGrow = "";
      for (int outerIndex = 0; outerIndex < 9; outerIndex++) {
        for (int innerIndex = 0; innerIndex < 1000; innerIndex++) {
          stringBuilderAppend.Append("Just some text to fill stringbuffer with 01234567890qwertyuiopasdfghjklzxcvbnm");
          stringBuilderInsert.Insert(0, "Just some text to fill stringbuffer with 01234567890qwertyuiopasdfghjklzxcvbnm");
        }
        stringBuilderAppend.Clear();
        stringBuilderInsert.Clear();
        capacityGrow += outerIndex + ": " + stringBuilderAppend.Capacity + ", " + stringBuilderInsert.Capacity  + Environment.NewLine;
      }
      //check in the debugger capacityGrow to see the capacity used in each iteration
      System.Diagnostics.Debugger.Break();
    }
  }
}

Edit1: After some more searching, I found the suggestion to set Capacity to 0 after Clear(), like this:

stringBuilder.Clear();
stringBuilder.Capacity = 0;

This prevents the memory leak and resulting Out of Memory exception when using Insert() repeatedly, but it also defeats the purpose of reusing the StringBuilder, which is to prevent allocation and deallocation of memory to achieve a higher speed. Instead to set Capacity=0, one could as well just use a new StringBuilder instead of Clear().

It seems that Insert() is not reusing the unused memory assigned already to the StringBuilder, but keeps adding new memory, despite the Clear(). I still feel this is a bug, because who would expect to run out of memory despite clearing the StringBuilder ? I would appreciate some comments if this is a bug and if it should be reported to Microsoft.

Edit2: I reported the bug to Microsoft on: https://connect.microsoft.com/VisualStudio/feedback/details/1242873

Please follow the link and upvote the bug, otherwise I fear Microsoft will not look into it :-(

Closing remarks:

• From previous postings I made the experience that commenters quickly say there is a problem in my program. They comment then quite rudely, even if it is them who are wrong.

• From previous postings I made the experience that commenters like to mark a question as "This question already has an answer", just because they don't understand the difference between my question and the other questions. Indeed, there are many answered questions on stackoverlow for StringBuilder and Out of Memory. I read all I could find with google or stackoverlow. They are about APPEND() and about EXTREMELY long texts. They are not about Insert(). Please note that my problems occurs even with very short strings and only when using Insert(), but not when using Append() !

• From previous postings I made the experience that commenters like to suggest to use different programming, like avoiding using Insert() and solve the problem by using Append and adding the lines in inverted sequence. While this might be possible in some cases, I feel that Insert() and Clear() should work properly and we should Microsoft let know if there is a problem.

Answer 1

I would say that the behaviour is to be expected:

Clear()

The doc states:

Removes all characters from the current StringBuilder instance.[...] Calling the Clear method does not modify the current instance's Capacity or MaxCapacity property.see Documenation

So Clear() removes the content but keeps the memory allocated.

Insert()

Here the doc states that the previous content is shifted and the capacity adjusted as needed. (doc) This is not completely clear, but I assume that new memory capacity + new length is allocated and the rest copied over. Now. this may not be the optimal solution, but I figure that this is the way it is implemented. So the out-of-memory situation is to be expected.

---

Another suggestion: Always inserting at the first position is - regardless of the implementation - pretty expensive. If you need the result only once consider using a List<string> where you insert the new lines at position 0 and then iterate over the list once and build the string.

---

Edit:

After looking at the (assumed) source I think that StringBuilder is indeed allocatign a new instance as 'chunk' for every insert - regardless of any other romm available. I followed the calls down to MakeRoom where at line 2044 a new chunk is allocated.

Answer 2

We ran into the same issue with

Replace()

when we updated our web app from .net 2 to .net 4.

Apparently the implementation of StringBuilder changed. See How the StringBuilder class is implemented? Does it internally create new string objects each time we append?

The solutions we looked at:

1. Do not reuse StringBuilder (that's what we decided to do after a big rewrite)
2. Write our own implementation of SB
3. Use .NET 2.0