Efficiently detect sign-changes in python

Question

I want to do exactly what this guy did:

Python - count sign changes

However I need to optimize it to run super fast. In brief I want to take a time series an

Answer 1

Another way to count zero crossings and squeeze just a few more milliseconds out of the code is to use nonzero and compute the signs directly. Assuming you have a one-dimensional array of data:

def crossings_nonzero_all(data):
    pos = data > 0
    npos = ~pos
    return ((pos[:-1] & npos[1:]) | (npos[:-1] & pos[1:])).nonzero()[0]

Alternatively, if you just want to count the zero crossings for a particular direction of crossing zero (e.g., from positive to negative), this is even faster:

def crossings_nonzero_pos2neg(data):
    pos = data > 0
    return (pos[:-1] & ~pos[1:]).nonzero()[0]

On my machine these are a bit faster than the where(diff(sign)) method (timings for an array of 10000 sine samples containing 20 cycles, 40 crossings in all):

$ python -mtimeit 'crossings_where(data)'
10000 loops, best of 3: 119 usec per loop

$ python -mtimeit 'crossings_nonzero_all(data)'
10000 loops, best of 3: 61.7 usec per loop

$ python -mtimeit 'crossings_nonzero_pos2neg(data)'
10000 loops, best of 3: 55.5 usec per loop

Answer 2

Do you want to time it? Or do you want to make it as fast as possible?

Timing is easy. Run it a zillion times, stopwatch it, and divide by a zillion.

To make it as fast as possible, what you need to do is find out what's taking time and that you could do in a better way. I use either 1) the random-pause technique, or 2) the single-step technique.

Answer 3

Jim Brissom's answer fails if a contains the value 0:

import numpy  
a2 = [1, 2, 1, 1, 0, -3, -4, 7, 8, 9, 10, -2, 1, -3, 5, 6, 7, -10]  
zero_crossings2 = numpy.where(numpy.diff(numpy.sign(a2)))[0]  
print zero_crossings2  
print len(zero_crossings2)  # should be 7

Output:

[ 3  4  6 10 11 12 13 16]  
8  

The number of zero crossing should be 7, but because sign() returns 0 if 0 is passed, 1 for positive, and -1 for negative values, diff() will count the transition containing zero twice.

An alternative might be:

a3 = [1, 2, 1, 1, 0, -3, -4, 7, 8, 9, 10, 0, -2, 0, 0, 1, 0, -3, 0, 5, 6, 7, -10]  
s3= numpy.sign(a3)  
s3[s3==0] = -1     # replace zeros with -1  
zero_crossings3 = numpy.where(numpy.diff(s3))[0]  
print s3  
print zero_crossings3  
print len(zero_crossings3)   # should be 7

which give the correct answer of:

[ 3  6 10 14 15 18 21]
7

Answer 4

Another way that might suit certain applications is to extend the evaluation of the expression np.diff(np.sign(a)).

If we compare how this expression reacts to certain cases:

1. Rising crossing without zero: np.diff(np.sign([-10, 10])) returns array([2])
2. Rising crossing with zero: np.diff(np.sign([-10, 0, 10])) returns array([1, 1])
3. Falling crossing without zero: np.diff(np.sign([10, -10])) returns array([-2])
4. Falling crossing with zero: np.diff(np.sign([10, 0, -10])) returns array([-1, -1])

So we have to evaluate np.diff(...) for the returned patterns in 1. and 2:

sdiff = np.diff(np.sign(a))
rising_1 = (sdiff == 2)
rising_2 = (sdiff[:-1] == 1) & (sdiff[1:] == 1)
rising_all = rising_1
rising_all[1:] = rising_all[1:] | rising_2

and for the cases 3. and 4.:

falling_1 = (sdiff == -2) #the signs need to be the opposite
falling_2 = (sdiff[:-1] == -1) & (sdiff[1:] == -1)
falling_all = falling_1
falling_all[1:] = falling_all[1:] | falling_2

After this we can easily find the indices with

indices_rising = np.where(rising_all)[0]
indices_falling = np.where(falling_all)[0]
indices_both = np.where(rising_all | falling_all)[0]

This approach should be reasonable fast because it can manage without using a "slow" loop.

This combines the approach of several other answers.

Answer 5

What about:

import numpy
a = [1, 2, 1, 1, -3, -4, 7, 8, 9, 10, -2, 1, -3, 5, 6, 7, -10]
zero_crossings = numpy.where(numpy.diff(numpy.sign(a)))[0]

Output:

> zero_crossings
array([ 3,  5,  9, 10, 11, 12, 15])

I.e., zero_crossings will contain the indices of elements before which a zero crossing occurs. If you want the elements after, just add 1 to that array.

Answer 6

I see people using diff a lot in their solutions, but xor seems to be much faster and the result is the same for bools (a good pointer to that might also be the fact that using diff gives a deprecated warning.... :) ) Here is an example:

positive = a2 > 0
np.where(np.bitwise_xor(positive[1:], positive[:-1]))[0]

Time it measures it to be around one and a half faster to diff for me:)

If you do not care about edge cases it might be better to use

positive = np.signbit(a2)

but positive = a2 >0 seems faster (and cleaner) than signbit AND checking for 0s (e.g. positive = np.bitwise_or(np.signbit(a2),np.logical_not(a2)) is slower...)