I borrowed the following method from somewhere on the internet (Can\'t remember where). But its doing a straight forward process, finding the distance between two gps points
That's the haversine algorithm, will provide you with a decent level of accuracy.
If it really is "millions" of points, perhaps implement a cache of calculations that you've made... if you come across a pair of coordinates, both of which are sufficiently close to a pair whose distance you've already calculated, then use the cached value?
Or try to cache some of the intermediate steps, e.g. degree to radians conversions.