could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR

Question

I installed tensorflow 1.0.1 GPU version on my Macbook Pro with GeForce GT 750M. Also installed CUDA 8.0.71 and cuDNN 5.1. I am running a tf code that works fine with non C

Answer 1

In my case it seems that the problem was caused by tensorflow and cudnn version mismatch. The following helped me (I was working on Ubuntu 16.04 with NVidia Tesla K80 on Google Cloud, tensorflow 1.5 finally worked with cudnn 7.0.4 and cuda 9.0):

1. Remove cuDNN completely:

   sudo rm /usr/local/cuda/include/cudnn.h
   sudo rm /usr/local/cuda/lib64/libcudnn*
   

   After doing so import tensorflow should cause error.

2. Download appropriate cuDNN version. Note that there is cuDNN 7.0.4 for CUDA 9.0 and cuDNN 7.0.4 for CUDA 8.0. You should choose the one corresponding to your CUDA version. Be careful at this step or you'll get similar problem again. Install cuDNN as usual:

   tar -xzvf cudnn-9.0-linux-x64-v7.tgz
   cd cuda
   sudo cp -P include/cudnn.h /usr/include
   sudo cp -P lib64/libcudnn* /usr/lib/x86_64-linux-gnu/
   sudo chmod a+r /usr/lib/x86_64-linux-gnu/libcudnn*
   

   In this example I've installed cuDNN 7.0.x for CUDA 9.0 (x actually doesn't matter). Take care to match your CUDA version.

3. Restart the computer. In my case the problem vanished. If the error still occurs consider installing another version of tensorflow.

Hope this helps someone.