When can Java produce a NaN?

Question

I know what Java Double.NaN is. I have some Java code that produces NaN.

// calculate errors
delta = m1 + m2 - M;
eta = f1 + f2          


        

Answer 1

NaN is triggered by the following occurrences:

• results that are complex values
  • √x where x is negative
  • log(x) where x is negative
  • tan(x) where x mod 180 is 90
  • asin(x) or acos(x) where x is outside [-1..1]
• 0/0
• ∞/∞
• ∞/−∞
• −∞/∞
• −∞/−∞
• 0×∞
• 0×−∞
• 1^∞
• ∞ + (−∞)
• (−∞) + ∞

Sorry for such a general answer, but I hope that helped.

Answer 2

According to Wikipedia:

There are three kinds of operation which return NaN:

• Operations with a NaN as at least one operand
• Indeterminate forms
  • The divisions 0/0, ∞/∞, ∞/−∞, −∞/∞, and −∞/−∞
  • The multiplications 0×∞ and 0×−∞
  • The power 1^∞
  • The additions ∞ + (−∞), (−∞) + ∞ and equivalent subtractions.
• Real operations with complex results:
  • The square root of a negative number
  • The logarithm of a negative number
  • The tangent of an odd multiple of 90 degrees (or π/2 radians)
  • The inverse sine or cosine of a number which is less than −1 or greater than +1.

This Java snippet illustrates all of the above, except the tangent one (I suspect because of limited precision of double):

import java.util.*;
import static java.lang.Double.NaN;
import static java.lang.Double.POSITIVE_INFINITY;
import static java.lang.Double.NEGATIVE_INFINITY;

public class NaN {
    public static void main(String args[]) {
        double[] allNaNs = {
            0D/0D,
            POSITIVE_INFINITY / POSITIVE_INFINITY,
            POSITIVE_INFINITY / NEGATIVE_INFINITY,
            NEGATIVE_INFINITY / POSITIVE_INFINITY,
            NEGATIVE_INFINITY / NEGATIVE_INFINITY,
            0 * POSITIVE_INFINITY,
            0 * NEGATIVE_INFINITY,
            Math.pow(1, POSITIVE_INFINITY),
            POSITIVE_INFINITY + NEGATIVE_INFINITY,
            NEGATIVE_INFINITY + POSITIVE_INFINITY,
            POSITIVE_INFINITY - POSITIVE_INFINITY,
            NEGATIVE_INFINITY - NEGATIVE_INFINITY,
            Math.sqrt(-1),
            Math.log(-1),
            Math.asin(-2),
            Math.acos(+2),
        };
        System.out.println(Arrays.toString(allNaNs));
        // prints "[NaN, NaN...]"
        System.out.println(NaN == NaN); // prints "false"
        System.out.println(Double.isNaN(NaN)); // prints "true"
    }
}

References

• Wikipedia/NaN
• JLS 15.21.1 Numerical Equality Operators == and !=

  If either operand is NaN, then the result of == is false but the result of != is true. Indeed, the test x!=x is true if and only if the value of x is NaN. (The methods Float.isNaN and Double.isNaN may also be used to test whether a value is NaN.)

Answer 3

Given what I know about gradient descent, you are most probably jumping out to infinity because you do not have adaptive rate (i.e. your rate is too big).

Answer 4

Have you tried sprinkling your code with System.out.println statements to determine exactly where NaNs start occuring?