Problems with LD_PRELOAD and calloc() interposition for certain executables

Question

Relating to a previous question of mine

I\'ve successfully interposed malloc, but calloc seems to be more problematic.

That is with ce

Answer 1

I know I am a bit late (6 years). But I wanted to override calloc() today and faced a problem because dlsym() internally uses calloc(). I solved it using a simple technique and thought of sharing it here:

static unsigned char buffer[8192];

void *calloc(size_t nmemb, size_t size)
{
    if (calloc_ptr == NULL) // obtained from dlsym
            return buffer;

    init(); // uses dlsym() to find address of the real calloc()

    return calloc_ptr(len);
}

void free(void *in)
{
    if (in == buffer)
        return;

    free_ptr(in);
}

buffer satisfies the need of dlsym() till the real calloc() has been located and my calloc_ptr function pointer initialized.

Answer 2

Using dlsym based hooking can result in crashes, as dlsym calls back into the memory allocator. Instead use malloc hooks, as I suggested in your prior question; these can be installed without actually invoking dlsym at all.

Answer 3

With regard to __nano_init() being called twice: You've declared the function as a constructor, so it's called when the library is loaded, and it's called a second time explicitly when your malloc() and calloc() implementations are first called. Pick one.

With regard to the calloc() interposer crashing your application: Some of the functions you're using, including dlsym() and fprintf(), may themselves be attempting to allocate memory, calling your interposer functions. Consider the consequences, and act accordingly.

Answer 4

You can get away with a preliminary poor calloc that simply returns NULL. This actually works on Linux, YMMV.

static void* poor_calloc(size_t nmemb, size_t size)
{
    // So dlsym uses calloc internally, which will lead to infinite recursion, since our calloc calls dlsym.
    // Apparently dlsym can cope with slightly wrong calloc, see for further explanation:
    // http://blog.bigpixel.ro/2010/09/interposing-calloc-on-linux
    return NULL; // This is a poor implementation of calloc!
}