Math optimization in C#

Question

I\'ve been profiling an application all day long and, having optimized a couple bits of code, I\'m left with this on my todo list. It\'s the activation function for a neural

Answer 1

Remember, Sigmoid constraints results to range between 0 and 1. Values of smaller than about -10 return a value very, very close to 0.0. Values of greater than about 10 return a value very, very close to 1.

Back in the old days when computers couldn't handle arithmetic overflow/underflow that well, putting if conditions to limit the calculation was usual. If I were really concerned about its performance (or basically Math's performance), I would change your code to the old fashioned way (and mind the limits) so that it does not call Math unnecessarily:

public double Sigmoid(double value)
{
    if (value < -45.0) return 0.0;
    if (value > 45.0) return 1.0;
    return 1.0 / (1.0 + Math.Exp(-value));
}

I realize anyone reading this answer may be involved in some sort of NN development. Be mindful of how the above affects the other aspects of your training scores.