MATLAB crashes when unloading mex file which has used CUDA memory

问题

I have been trying to figure this out for quite some time.

I use a MEX file in matlab (Linux 64bit) which uses CUDA. The code compiles and executes fine but when I want to unload the mex (e.g. to recompile it or when matlab exits), matlab crashes immediately without any message and with an empty dump.

I was able reduce it to a minimal working example:

MEX cpp File:

#include <stdint.h>
#include "mex.h"

extern "C" void cudaTest();

void mexFunction(
                int nlhs, mxArray *plhs[],
                int nrhs, const mxArray *prhs[])
{
    cudaTest();
}

CUDA File compiled with NVCC:

void cudaTest() {

    float* d_test = NULL;
    cudaMalloc((void**) &d_test, 10000 * sizeof(float));

    cudaFree(d_test);
}

While with my real program it always crashes, with this minimal example it is not always reproducible. Sometimes it does crash sometimes not..

回答1:

I think this solved my problem:

http://www.mathworks.de/matlabcentral/answers/45307

回答2:

Hmm, It may be memory problem which your forgot to free.

Some suggestions might useful:

• Don't use MATLAB memory management function: mxalloc..., outside mexfunction or matlab wrap, your mex function might run some process background and might cause MATLAB crash, when mex function call memory management function simultaneously with matlab.

• register mexAtExit(clearfunction) function(see MATLAB help: mexAtExit) clear your mex memory and thread which is not managed by MATLAB aotumaticly, i.e. cudaMalloc here. when mex function unload or matlab exit, MATLAB would automaticly clear mexfunction. so if your momery management function is not MATLAB memory management function, MATLAB wouldn't know how to deal with your mex program.

• debug your function as below

run:

clear your_mex_function

MATLAB would call clearfunction(this function is a mexatexit register function see upside step) of your_mex_function, and you will find out what's the problem of your mex function.

来源：https://stackoverflow.com/questions/13415291/matlab-crashes-when-unloading-mex-file-which-has-used-cuda-memory