returning std::string/std::list from dll

Question

Short question.

I just got a dll I\'m supposed to interface with. Dll uses crt from msvcr90D.dll (notice D), and returns std::strings, std::lists, and boost::shared_

Answer 1

I have this exact problem in a project I'm working on - STL classes are transmitted to and from DLLs a lot. The problem isn't just the different memory heaps - it's actually that the STL classes have no binary standard (ABI). For example, in debug builds, some STL implementations add extra debugging information to the STL classes, such that sizeof(std::vector) (release build) != sizeof(std::vector) (debug build). Ouch! There's no hope you can rely on binary compatibility of these classes. Besides, if your DLL was compiled in a different compiler with some other STL implementation that used other algorithms, you might have different binary format in release builds, too.

The way I've solved this problem is by using a template class called pod (POD stands for Plain Old Data, like chars and ints, which usually transfer fine between DLLs). The job of this class is to package its template parameter in to a consistent binary format, and then unpackage it at the other end. For example, instead of a function in a DLL returning a std::vector, you return a pod>. There's a template specialization for pod>, which mallocs a memory buffer and copies the elements. It also provides operator std::vector(), so that the return value can transparently be stored back in to a std::vector, by constructing a new vector, copying its stored elements in to it, and returning it. Because it always uses the same binary format, it can be safely compiled in to separate binaries and remain binary compatible. An alternative name for pod could be make_binary_compatible.

Here's the pod class definition:

// All members are protected, because the class *must* be specialization
// for each type
template
class pod {
protected:
    pod();
    pod(const T& value);
    pod(const pod& copy);                   // no copy ctor in any pod
    pod& operator=(const pod& assign);
    T get() const;
    operator T() const;
    ~pod();
};

Here's the partial specialization for pod> - note, partial specialization is used so this class works for any type of T. Also note, it actually is storing a memory buffer of pod rather than just T - if the vector contained another STL type like std::string, we'd want that to be binary compatible too!

// Transmit vector as POD buffer
template
class pod > {
protected:
    pod(const pod >& copy);  // no copy ctor

    // For storing vector as plain old data buffer
    typename std::vector::size_type  size;
    pod*                             elements;

    void release()
    {
        if (elements) {

            // Destruct every element, in case contained other cr::pods
            pod* ptr = elements;
            pod* end = elements + size;

            for ( ; ptr != end; ++ptr)
                ptr->~pod();

            // Deallocate memory
            pod_free(elements);
            elements = NULL;
        }
    }

    void set_from(const std::vector& value)
    {
        // Allocate buffer with room for pods of T
        size = value.size();

        if (size > 0) {
            elements = reinterpret_cast*>(pod_malloc(sizeof(pod) * size));

            if (elements == NULL)
                throw std::bad_alloc("out of memory");
        }
        else
            elements = NULL;

        // Placement new pods in to the buffer
        pod* ptr = elements;
        pod* end = elements + size;
        std::vector::const_iterator iter = value.begin();

        for ( ; ptr != end; )
            new (ptr++) pod(*iter++);
    }

public:
    pod() : size(0), elements(NULL) {}

    // Construct from vector
    pod(const std::vector& value)
    {
        set_from(value);
    }

    pod >& operator=(const std::vector& value)
    {
        release();
        set_from(value);
        return *this;
    }

    std::vector get() const
    {
        std::vector result;
        result.reserve(size);

        // Copy out the pods, using their operator T() to call get()
        std::copy(elements, elements + size, std::back_inserter(result));

        return result;
    }

    operator std::vector() const
    {
        return get();
    }

    ~pod()
    {
        release();
    }
};

Note the memory allocation functions used are pod_malloc and pod_free - these are simply malloc and free, but using the same function between all DLLs. In my case, all DLLs use the malloc and free from the host EXE, so they are all using the same heap, which solves the heap memory issue. (Exactly how you figure this out is down to you.)

Also note you need specializations for pod, pod, and pod for all the basic types (pod, pod etc), so that they can be stored in a "pod vector" and other pod containers. These should be straightforward enough to write if you understand the above example.

This method does mean copying the entire object. You can, however, pass references to pod types, since there is an operator= which is safe between binaries. There's no real pass-by-reference, though, since the only way to change a pod type is to copy it out back to its original type, change it, then repackage as a pod. Also, the copies it creates mean it's not necessarily the fastest way, but it works.

However, you can also pod-specialize your own types, which means you can effectively return complex types like std::map> providing there's a specialization for pod and partial specializations for std::map, std::vector and std::basic_string (which you only need to write once).

The end result usage looks like this. A common interface is defined:

class ICommonInterface {
public:
    virtual pod> GetListOfStrings() const = 0;
};

A DLL might implement it as such:

pod> MyDllImplementation::GetListOfStrings() const
{
    std::vector ret;

    // ...

    // pod can construct itself from its template parameter
    // so this works without any mention of pod
    return ret;
}

And the caller, a separate binary, can call it as such:

ICommonInterface* pCommonInterface = ...

// pod has an operator T(), so this works again without any mention of pod
std::vector list_of_strings = pCommonInterface->GetListOfStrings();

So once it's set up, you can use it almost as if the pod class wasn't there.