What is the fastest way to convert string to array in python?

Question

This is a line I read from a text file:

[54, 95, 45, -97, -51, 84, 0, 32, -55, 14, 50, 54, 68, -3, 57, 88, -1]

I used readline() to read it

Answer 1

Since we care about speed, in this particular case I might use json.loads:

>>> import ast, json
>>> s = "[54, 95, 45, -97, -51, 84, 0, 32, -55, 14, 50, 54, 68, -3, 57, 88, -1]"
>>> %timeit ast.literal_eval(s)
10000 loops, best of 3: 61.6 µs per loop
>>> %timeit eval(s)
10000 loops, best of 3: 45.7 µs per loop
>>> %timeit json.loads(s)
100000 loops, best of 3: 6.61 µs per loop
>>> json.loads(s)
[54, 95, 45, -97, -51, 84, 0, 32, -55, 14, 50, 54, 68, -3, 57, 88, -1]

(Note that this works here because this line is sufficiently JSON-like. It can't be used everywhere that ast.literal_eval can because not all Python literal syntax is valid JSON.)

Answer 2

I'm using python 3.6

Here's my result of an actual string separated by spaces

stringValue = "123 456 789 012 345 678"

String to List

intValue = list(map(int, stringValue.split(' ')))

result

time taken 7.586999345221557e-06

[76, 96, 127, 255, 136, 164]

time taken trying to print it 6.697199933114462e-05

String to List to numpy

intValue = np.array(list(map(int, stringValue.split(' '))))

result

time taken 2.631999996083323e-05

[ 76 96 127 255 136 164]

time taken trying to print it 0.002241893000245909

String to numpy array

intValue = np.fromstring(stringValue,dtype=int,sep=' ')

time taken 1.3830000170855783e-05

[ 76 96 127 255 136 164]

time taken trying to print it 0.0003395890007595881

Answer 3

I'm not sure that this is the fastest, but it's definitely the safest/easiest:

import ast
lst = ast.literal_eval(s)

regular eval would work too:

lst = eval(s)

Some basic timings from my machine:

>>> s = '[54, 95, 45, -97, -51, 84, 0, 32, -55, 14, 50, 54, 68, -3, 57, 88, -1]' 
>>> def f1():
...    eval(s)
... 
>>> def f2():
...    ast.literal_eval(s)
... 
>>> timeit.timeit('f1()', 'from __main__ import f1')
31.415852785110474
>>> timeit.timeit('f2()', 'from __main__ import f2')
46.25958704948425

So, according to my computer, eval is about 50% faster than ast.literal_eval. However, eval is terribly unsafe and should never be used on any string unless you trust it completely. Unless this is a real demonstratable bottleneck and you trust the input 100%, I would consider the little bit of extra time worth it in exchange for being able to sleep soundly at night.

Answer 4

If that's the string, go here http://docs.python.org/2/library/functions.html#eval

>>> s = "[54, 95, 45, -97, -51, 84, 0, 32, -55, 14, 50, 54, 68, -3, 57, 88, -1]"
>>> eval(s)
[54, 95, 45, -97, -51, 84, 0, 32, -55, 14, 50, 54, 68, -3, 57, 88, -1]