Cost function in logistic regression gives NaN as a result

Question

I am implementing logistic regression using batch gradient descent. There are two classes into which the input samples are to be classified. The classes are 1 and 0. While t

Answer 1

It happened to me because an indetermination of the type:

0*log(0)

This can happen when one of the predicted values Y equals either 0 or 1. In my case the solution was to add an if statement to the python code as follows:

y * np.log (Y)  + (1-y) * np.log (1-Y) if ( Y != 1 and Y != 0 ) else 0

This way, when the actual value (y) and the predicted one (Y) are equal, no cost needs to be computed, which is the expected behavior.

(Notice that when a given Y is converging to 0 the left addend is canceled (because of y=0) and the right addend tends toward 0. The same happens when Y converges to 1, but with the opposite addend.)

(There is also a very rare scenario, which you probably won't need to worry about, where y=0 and Y=1 or viceversa, but if your dataset is standarized and the weights are properly initialized it won't be an issue.)