Is it possible to track down which expression caused an NPE?

Question

When I get an NPE, I'll get a stack trace with line number. That's helpful, but if the line is very dense and/or contains nested expression, it's still impossible to figure out which reference was null.

Surely, this information must've been available somewhere. Is there a way to figure this out? (If not java expression, then at least the bytecode instruction that caused NPE would be helpful as well)

Edit #1: I've seen a few comments suggesting breaking up the line, etc, which, no offence, is really non-constructive and irrelevant. If I could do that, I would have ! Let just say this modifying the source is out of the question.

Edit #2: apangin has posted an excellent answer below, which I accepted. But it's SOOO COOL that I had to include the output here for anyone who doesn't want to try it out themselves! ;)

So suppose I have this driver program TestNPE.java

 1  public class TestNPE {
 2      public static void main(String[] args) {
 3          int n = 0;
 4          String st = null;
 5  
 6          System.out.println("about to throw NPE");
 7          if (n >= 0 && st.isEmpty()){
 8              System.out.println("empty");
 9          }
10          else {
11              System.out.println("othereise");
12          }
13      }
14      
15  }

The bytecode looks like this (showing only the main() method and omitting other irrelevant parts)

Code:
  stack=2, locals=3, args_size=1
     0: iconst_0
     1: istore_1
     2: aconst_null
     3: astore_2
     4: getstatic     #2                  // Field java/lang/System.out:Ljava/io/PrintStream;                                              
     7: ldc           #3                  // String about to throw NPE                                                                     
     9: invokevirtual #4                  // Method java/io/PrintStream.println:(Ljava/lang/String;)V                                      
    12: iload_1
    13: iflt          34
    16: aload_2
    17: invokevirtual #5                  // Method java/lang/String.isEmpty:()Z                                                           
    20: ifeq          34
    23: getstatic     #2                  // Field java/lang/System.out:Ljava/io/PrintStream;                                              
    26: ldc           #6                  // String empty                                                                                  
    28: invokevirtual #4                  // Method java/io/PrintStream.println:(Ljava/lang/String;)V                                      
    31: goto          42
    34: getstatic     #2                  // Field java/lang/System.out:Ljava/io/PrintStream;                                              
    37: ldc           #7                  // String othereise                                                                              
    39: invokevirtual #4                  // Method java/io/PrintStream.println:(Ljava/lang/String;)V                                      
    42: return

Now when you run the TestNPE driver with the agent, you'll get this

$ java -agentpath:libRichNPE.o TestNPE
about to throw NPE
Exception in thread "main" java.lang.NullPointerException: location=17
    at TestNPE.main(TestNPE.java:7)

So that points to the invokevirtual #5 at offset 17! Just HOW COOL IS THAT?

Answer 1

When an exception happens, JVM knows the original bytecode that caused the exception. However, StackTraceElement does not track bytecode indices.

The solution is to capture bytecode index using JVMTI whenever exception occurs.

The following sample JVMTI agent will intercept all exceptions, and if exception type is NullPointerException, the agent will replace its detailMessage with the bytecode location information.

#include <jvmti.h>
#include <stdio.h>

static jclass NullPointerException;
static jfieldID detailMessage;

void JNICALL VMInit(jvmtiEnv* jvmti, JNIEnv* env, jthread thread) {
    jclass localNPE = env->FindClass("java/lang/NullPointerException");
    NullPointerException = (jclass) env->NewGlobalRef(localNPE);

    jclass Throwable = env->FindClass("java/lang/Throwable");
    detailMessage = env->GetFieldID(Throwable, "detailMessage", "Ljava/lang/String;");
}

void JNICALL ExceptionCallback(jvmtiEnv* jvmti, JNIEnv* env, jthread thread,
                               jmethodID method, jlocation location, jobject exception,
                               jmethodID catch_method, jlocation catch_location) {
    if (env->IsInstanceOf(exception, NullPointerException)) {
        char buf[32];
        sprintf(buf, "location=%ld", (long)location);
        env->SetObjectField(exception, detailMessage, env->NewStringUTF(buf));
    }
}

JNIEXPORT jint JNICALL Agent_OnLoad(JavaVM* vm, char* options, void* reserved) {
    jvmtiEnv* jvmti;
    vm->GetEnv((void**)&jvmti, JVMTI_VERSION_1_0);

    jvmtiCapabilities capabilities = {0};
    capabilities.can_generate_exception_events = 1;
    jvmti->AddCapabilities(&capabilities);

    jvmtiEventCallbacks callbacks = {0};
    callbacks.VMInit = VMInit;
    callbacks.Exception = ExceptionCallback;
    jvmti->SetEventCallbacks(&callbacks, sizeof(callbacks));
    jvmti->SetEventNotificationMode(JVMTI_ENABLE, JVMTI_EVENT_VM_INIT, NULL);
    jvmti->SetEventNotificationMode(JVMTI_ENABLE, JVMTI_EVENT_EXCEPTION, NULL);

    return 0;
}

Compile this into a shared library and run java with -agentpath option:

java -agentpath:/pato/to/libRichNPE.so Main

Answer 2

The exception itself does not have enough information to provide more than line numbers.

One option i see is to use a bytecode debugger like bytecode visualizer to closer localize the bytecode instruction that causes the npe. Step forward until the exception occurs, or add a breakpoint for npe.

Answer 3

The stack trace mechanism relies on the debugging metadata optionally compiled into each class (namely the SourceFile and LineNumberTable attributes). As far as I know, bytecode offsets are not preserved anywhere. However, these would not be useful for a typical Java program, since you still have know what code each bytecode instruction corresponds to.

However, there is an obvious workaround - just break the code in question up into multiple lines and recompile! You can insert whitespace almost anywhere in Java.

Answer 4

You can either break up the complex line into many smaller ones you can trace, or you use your debugger to see what value was null when the exception occurred.

While you could try to look at the byte code where this happened, this will only be the start of a complex journey. I suggest making your code simpler to understand and you might work out which values could be null (Note: it could be null unless you know it's not possible)