Fast sigmoid algorithm

Question

The sigmoid function is defined as

I found that using the C built-in function exp() to calculate the value of f(x) is slow. Is th

Answer 1

To do the NN more flexible usually used some alpha rate to change the angle of graph around 0.

The sigmoid function looks like:

f(x) = 1 / ( 1+exp(-x*alpha))

The nearly equivalent, (but more faster function) is:

f(x) = 0.5 * (x * alpha / (1 + abs(x*alpha))) + 0.5

You can check the graphs here

When I using abs function the network become faster 100+ times.