Python calculate lots of distances quickly

Question

I have an input of 36,742 points which means if I wanted to calculate the lower triangle of a distance matrix (using the vincenty approximation) I would need to generate 36,742*

Answer 1

Have you tried mapping entire arrays and functions instead of iterating through them? An example would be as follows:

from numpy.random import rand

my_array = rand(int(5e7), 1)  # An array of 50,000,000 random numbers in double.

Now what is normally done is:

squared_list_iter = [value**2 for value in my_array]

Which of course works, but is optimally invalid.

The alternative would be to map the array with a function. This is done as follows:

func = lambda x: x**2  # Here is what I want to do on my array.

squared_list_map = map(func, test)  # Here I am doing it!

Now, one might ask, how is this any different, or even better for that matter? Since now we have added a call to a function, too! Here is your answer:

For the former solution (via iteration):

1 loop: 1.11 minutes.

Compared to the latter solution (mapping):

500 loop, on average 560 ns. 

Simultaneous conversion of a map() to list by list(map(my_list)) would increase the time by a factor of 10 to approximately 500 ms.

You choose!