Python and read text file that contains lists with strings and numbers

Question

This is a similar question to many but not quite the same. I have a text file that has about 400,000 lines of text. Each line is essentially a list. For example it looks like [ 'a','b',1,2,'c',3,'d and , e string', 45]

I can read each line of the text file with the following code:

with open('myfile.txt') as f:
    content = f.readlines()

The problem is that each line is read as a string. I would like to get each item of the list. So i thought i would do (for each line):

content[line].split(',')

This almost works, but i run into a problem. Many times in my text file i have a string in the list that has a comma in it (from above i had 'd and , e string'). I don't want this string split up but want it as one item.

If this doesn't make since I want to take the line from my text file [ 'a','b',1,2,'c',3,'d and , e string', 45] and i want 8 separate elements

'a'
'b'
1 
2
'c'
3
'd and , e string'
45

Thanks for the help!

Answer 1

DSM right

from ast import literal_eval

# use generator 
# instead of allocating 
# 400 000 lists at a time
your_lists = (literal_eval(s) for s in open('filename.txt'))