System.OutOfMemoryException when generating permutations

Question

I\'m getting System.OutOfMemoryException when trying to generate 6 letter permutations. 5 letter permutations still work.

Here is the code I\'m using to

Answer 1

Here's a simple solution that is both computationally and memory efficient.

• Instead of generating the entire list of permutations and then find matches, using an iterator lets us process potential permutation matches as they get generated.
• With a little bit of backtracking, only permutations that have a chance to match your regex get generated.

All you need is an extra regular expression which accepts partial candidates. It should accept strings that could become a match if characters get added. (It would have be nice to have something like hitEnd() in Java which does exactly this. This would eliminate the need for that regular expression. Unfortunately, I do not think there is an equivalent in .Net)

In my example, I want to find permutations of the string "123456789" that match regex "32145.67". I use the (suboptimal) regular expression "^3$|^32$|^321" to discard permutations that do not start with 321. (Of course, here it would have been possible to generate the permutations for "456789" and prepend "123" to the results, but this is just to illustrate the concept.)

The efficiency of this solution will rely mostly on how many invalid matches you can discard early in the generation of permutations.

Short explanation of how the permutation generation works. Let's try to generate all the permutations of the string "abc". It can easily be seen that:

permutations("abc") = {"a" + permutations("bc"),
                       "b" + permutations("ac"),
                       "c" + permutations("ab")}

In other words, we take each character of the input string, append it to an accumulator and compute all the permutations for the input string with the character removed. Once we reach a leaf - a permutation of the input string -, the accumulator will have the size of the input string.

This can be written succinctly in recursive pseudo-code as:

permutation(input, acc)
  if input empty
     return acc

  foreach(character in input)
      left <- remove char from input
      permutation(left, acc+char)

Now this is not the most efficient way to generate permutations. (see Heap's algorithm) But at least it allows us not to explore the entire tree structure and discard permutations just by looking at their prefix.

Since "yield return" does not work so well in recursive functions, I have simply rewritten the solution in a iterative manner (Note: space complexity is worse than with the above recursive DFS).

public IEnumerable getPermutation(string input, string regexp)
{
        Stack left = new Stack();
        Stack acc = new Stack();

        left.Push(input);
        acc.Push("");

        // generate all permutations that match regexp
        while (left.Count > 0)
        {
            string c = left.Pop();
            string r = acc.Pop();

            if(r.Length==input.Length)
            {
                yield return r;
            }
            else
            {
                for(int i=0;i