Mex function not updated after recompile

Question

I have a simple mex function, which calls another C++ function from a library. I compile the source with

mex -cxx mymexfunction.cpp -I/some/include -L/some/lib -lmylib

The mylib library is dynamic (.so) and is linked itself against some other libraries (boost, OpenCV and some more).

The problem I am having is that once I have called the function mymexfunction once, it will not get updated when I recompile the source code. I have tried

clear
clear all
clear mex
clear functions
clear fun('mymexfunction')
munlock('mymexfunction')
unloadlibrary('mymexfunction')

...but nothing helps! I have to restart Matlab in order to be able to see the updated mexfunction. Even if I delete the compiled mex file and recompile, I still get the old version of the mex function (not on disk, but in memory).

All is fine if I don't link against mylib, but I have no idea what could be the culprit that prevents the update. The library is unfortunately too large and too interwoven to remove single modules one by one.

Are there some known conditions which could cause such problems?

Clarification:

I only update the content of the mex function, not the library.

Update:

It works under Ubuntu 11.04 with Matlab R2011a! I tried to reproduce the same environment on my OpenSUSE machine (R2011a, Boost 1.42, OpenCV 2.2 dynamically linked, ...) but still no luck. So I conclude that nothing is actually wrong with my library (otherwise it wouldn't work under Ubuntu) but it must be some collision of the dependencies and Matlab internal libraries. I officially give up. Praetorian and Amro, thank you for your help!

Answer 1

The mex command automatically clears the mex function if it is currently loaded in memory. Are you sure your mex function is closing whatever handle it holds to the other library? If such a handle exists it might prevent the OS from unloading the mex file.

I've used the following set of commands to clear mex functions manually, and from my experience, using a full path to the mex file when calling clear works. So give this a try and if it still doesn't get unloaded you might want to start looking at the code for loading and unloading the other library.

[~,f] = inmem( '-completenames' );
result = strfind( f, ['mymexfile' '.' mexext] );
result = f(cellfun( @isempty, result, 'UniformOutput', true ) == 0);
clear( result{:} )

Try running the inmem command again after the above and see if your mex file is still listed.

Something that might help you with making sure the other library gets unloaded is maybe using an std::shared_ptr to hold the handle to this library. Then, at the beginning of the mexFunction() entry point load the library and stick the handle into the shared_ptr. The shared_ptr will also need to use a custom deleter to unload the library (on Windows the custom deleter would call FreeLibrary).

Of course, if this is being caused by a bug in the other library none of this is going to help.

Answer 2

Trying to reproduce the problem, I've written a minimum working example for your case: a MEX file that links against a dynamic library and use use one of its exposed functions. I've tested the following on WinXP 32-bit using MATLAB R2010b with VS2010 as compiler (for both the DLL and MEX).

The example simply adds floating-point numbers. The MEX file accepts matrices/vectors and loops over the elements calling the add() function from the library on each pair.

Adder.h

#ifndef ADDER_H
#define ADDER_H

#ifdef __cplusplus
extern "C" {
#endif

#ifdef _WIN32
#   ifdef BUILDING_DLL
#       define DLL_IMPORT_EXPORT __declspec(dllexport)
#   else
#       define DLL_IMPORT_EXPORT __declspec(dllimport)
#   endif
#else
#   define DLL_IMPORT_EXPORT
#endif

DLL_IMPORT_EXPORT double add(double x, double y);

#ifdef __cplusplus
}
#endif

#endif

Adder.c

#include "Adder.h"

double add(double x, double y)
{
    return x+y;
}

mymexfunction.c

#include "mex.h"
#include "Adder.h"

#define X_IN    input[0]
#define Y_IN    input[1]
#define Z_OUT   output[0]

void mexFunction(int output_size, mxArray *output[], int input_size, const mxArray *input[])
{
    double *inX, *inY, *outZ;
    mwSize m,n;
    int i;

    /* check for proper number of arguments */
    if (input_size != 2) {
        mexErrMsgTxt("Two input arguments required.");
    }
    if (output_size > 1) {
        mexErrMsgTxt("Too many output arguments.");
    }

    /* check input argument sizes */
    m = mxGetM(X_IN);
    n = mxGetN(X_IN);
    if ( !mxIsDouble(X_IN) || !mxIsDouble(Y_IN) ) {
        mexErrMsgTxt("Input arguments must be matrices/vectors of doubles.");
    }
    if ( mxGetM(Y_IN)!=m || mxGetN(Y_IN)!=n ) {
        mexErrMsgTxt("X and Y must be of same size.");
    }

    /* Create a matrix for the return argument */
    Z_OUT = mxCreateDoubleMatrix(m, n, mxREAL);

    // get pointers to data
    inX =  (double *) mxGetPr(X_IN);
    inY =  (double *) mxGetPr(Y_IN);
    outZ = (double *) mxGetPr(Z_OUT);

    // compute and store result
    for(i=0; i<m*n; ++i) {
        outZ[i] = add(inX[i], inY[i]);
    }

    return;
}

First we build the dynamic library, as I mentioned I am using VC++ for the job. On Unix-based systems with GCC, I think this step goes like (correct me if I am wrong):

gcc -c -DBUILDING_DLL Adder.c -o Adder.o -I.
gcc -shared -o libAdder.so Adder.o -Wl,--out-implib,libAdder.a

then in MATLAB, we compile the MEX file:

>> mex mymexfunction.c -I. -L. -lAdder

(Note: I put everything in the same folder to avoid dealing with path issues.)

Next, we can test the function in MATLAB:

>> mymexfunction([1 2;3 4], [5 6; 7 8])
ans =
     6     8
    10    12

Using the Process Explorer tool from Sysinternals, we can view the loaded DLLs by the MATLAB process, the MEX function and our custom dynamic library:

If we issue the command clear mex then both modules are unloaded as expected (which is verified using Process Explorer). This is also confirmed by INMEM as @Praetorian showed:

clear mex
[~,m] = inmem('-completenames');
any( ismember(m,fullfile(pwd,['mymexfunction.' mexext])) )

Finally if we make some changes to mymexfunction.c:

// add 10 to all results
outZ[i] = add(inX[i], inY[i]) + 10.0;

recompile the MEX, and test it again (all in the same session, no restart). The result will reflect the changes made as you can see:

>> mymexfunction([1 2;3 4], [5 6; 7 8])
ans =
    16    18
    20    22

Please try to repeat the above on your Mac/Linux machine. If you still receive the old sums, then it must be a bug specific to non-Windows platforms, and should be reported to MathWorks... Otherwise I suspect that in your code, there must be some un-released resources causing the module to remain in memory?