This is what worked for me. I referred to the Tensorflow website to find the currently supported CudaNN version. When I solved my issue, it was CudaNN 6.x. The latest version, in my case 7 DLL is not loaded by tensorflow, and if you try to rename it, it causes memory errors.