1. 技术文章
  2. TensorFlow: Blas GEMM launch failed

TensorFlow: Blas GEMM launch failed

由 允我心安 提交于 2019-11-30 02:24:03

It's a simple fix, but it was a nightmare to figure it all out

On Windows I found the Keras install in Anaconda3\Lib\site-packages\keras

sources:

https://www.tensorflow.org/guide/using_gpu

https://github.com/keras-team/keras/blob/master/keras/backend/tensorflow_backend.py

Find the following in your keras/tensorflow_backend.py file you'll add config.gpu_options.allow_growth= True in both places 

if _SESSION is None:
            if not os.environ.get('OMP_NUM_THREADS'):
                config = tf.ConfigProto(allow_soft_placement=True)
                config.gpu_options.allow_growth=True
            else:
                num_thread = int(os.environ.get('OMP_NUM_THREADS'))
                config = tf.ConfigProto(intra_op_parallelism_threads=num_thread,
                                        allow_soft_placement=True)
                config.gpu_options.allow_growth=True
            _SESSION = tf.Session(config=config)
        session = _SESSION
fotis j

Had the same error. Maybe it is related to the problem that tensorflow is allocating all gpu memory. But the fix recommended there didn't work for me and it is not possible yet to limit tensorflow's gpu memory use via keras.json or commandline. Switching keras' backend to Theano resolved the issue for me (howto can be found here).

This Answer is much related to Tensorflow:

Sometimes Tensorflow fails at creation in Windows.

Restarting the notebook using gpu solves it in most cases

If it doesnt then try restarting the notebook after adding these options in your code.

gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=0.9)

tf.Session(config=tf.ConfigProto(gpu_options=gpu_options,allow_soft_placement=True)

I never had such error while using Keras But try restarting your notebook

liwy

Make sure you have no other processes using the GPU running. Run nvidia-smi to check this.

SOURCE: An issue brought up by @reedwm.

Try running the sample program simpleCUBLAS (it comes with CUDA) to test your CUBLAS installation and see if it works.

In my case (I am using Ubuntu) I had to reinstall CUDA to solve this issue. After I did that, simpleCUBLAS passed the test.

For some reason I started running into the same issue after a while, and I found that cleaning the directory .nv (inside my home folder) resolved the issue, and simpleCUBLAS test passed again.

Tairone

I was getting exactly the same error message. I realized that there was an error with my CUDA installation, specifically with the cuBLAS library.

You can check if yours has the same problem by running the sample program simpleCUBLAS (it comes with the CUDA installation, you will probably find it in the CUDA home folder:$CUDA_HOME\samples\7_CUDALibraries\simpleCUBLAS)

Try running this program. If the test fails, you have a problem with your CUDA installation. You should try to reinstall it. That's how I solved the same problem here.

xiaxia wang

I have got the same error,lucky,I have got it fixed. my error is: the last time,I open the tensorflow sess = tf.Session(),but I forgot close the session.

so I open the terminal, type command:

ps -aux | grep program_name

find the PID,and type command kill the PID:

kill -9 PID

Ok,the GPU is realase.

Had the same error (Win10 using Keras and Visual Studio Code). Seems like TensorFlow was still active somehow even after terminating my script. Simply closing VS Code and restarting solved the issue.

I ran into this problem when trying to run several servers that use a model to serve predictions. As I wasn't training a model but simply using it, the difference between using GPU or CPU was minor. For this specific case, the issue can be avoided by forcing Tensorflow to use the CPU by "hiding" the GPU.

import os
os.environ["CUDA_VISIBLE_DEVICES"] = "-1"  # Force TF to use only the CPU
标签
python
tensorflow
keras
blas
易学教程内所有资源均来自网络或用户发布的内容，如有违反法律规定的内容欢迎反馈！
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!