Using combinations in python for very large sequences

问题

I'm trying to determine all the combinations of 87 different strings that could make up a 29 element sequence. I was using combinations in python to do this and it works fine if the sequence is only 4 elements long but it can't handle 29. This is the code I'm using:

combos = itertools.combinations(testv, 29)

usable_combos = []
for i in combos:
    usable_combos.append(i)

but the code fails at the loop stage. I assume this is some kind of memory issue but I'm not sure how to fix it. Any suggestions?

回答1:

You are trying to stuff a huge number of tuples into a list here. 101.416.867.967.028.166.758.360 different tuples, to be precise. A number so big I don't even know how to spell it out, but you can start with 101 and almost a half sextillion as an approximation.

When you made combinations of 4 elements, there were just 2.225.895 different combinations (a little over 2 million), something that is quite manageable, but you pushed it to levels that most computers simply cannot store in memory all at once.

Rather than add everything to a list, then use that list, you'd be better off processing those combinations as you loop:

for i in combos:
     # process i, move on

or find a different approach to solving your problem, one that doesn't involve looping over all those possible combinations. Perhaps there are ways to reduce the number of combinations you actually need to consider?

As an aside, rather than use a for loop and list.append(), you could have just used:

combos = itertools.combinations(testv, 4)
usable_combos = list(combos)

to create your list.

来源：https://stackoverflow.com/questions/27931355/using-combinations-in-python-for-very-large-sequences