corrupted double-linked list?

Question

I am just debugging a big project and and getting this error

*** glibc detected *** p_appmanager/obj/appmanager: corrupted double-linked list: 0x08325e18 **         


        

Answer 1

It looks like a thread is printing data (char* character data) that was already freed on the main (or another) thread.

More specificly, Parser::parse builds a vector of std::strings.

---

EDIT As requested in the comment, this is what I read, with [highlights]:

==2181== [Invalid read of size 1]
==2181==at 0x402C658: strlen (in /usr/lib/valgrind/vg...
==2181==by 0x40FCC2D: std::basic_ostream<char, std::c...
==2181==by 0x4054D4B: start_thread (pthread_create.c:...
==2181==by 0x4269DDD: clone (clone.S:130)
==2181== [Address 0x4359234] is 12 bytes inside [a block of size 21 free'd]
==2181==at 0x402ACFC: operator delete(void*) (in /usr...
==2181==by 0x410899A: std::string::_Rep::_M_destroy(s...
==2181==by 0x4071B37: void std::_Destroy_aux<false>::...
==2181==by 0x40717DC: void std::_Destroy<std::string*...
==2181==by 0x4071041: void std::_Destroy<std::string*...
==2181== by 0x407088A: std::vector<std::string, std::a... [::~vector()] (in /usr/lib/libparser.so
==2181== by 0x406F497: [Parser::parse(std::string)] (in /usr/lib/libparser.so
==2181==by 0x40729E2: getMessage(int, Message**) (in ...
==2181==by 0x804CB99: fifoThread(void*) (in /home/cle...
==2181==by 0x4054D4B: start_thread (pthread_create.c:...
==2181==by 0x4269DDD: clone (clone.S:130

And I noticed the other backtracecs all highlighted the same pattern.

---

Apparently, some thread(s) get started, and they are passed references (in)to these strings. However, since the vector is local to Parser::parse (probably an automatic variable, or explicitely deleted), the references are no longer valid (e.g. at the time the strings are being streamed using std::ostream::operator<<(std::ostream&, const char*)).

Sadly, the stack trace doesn't quite show what name/object module contains the offending line, but you can see it lives on a separate thread (due to the presence of start_thread)

Edit Apparently, the thread(s) take a reference to the strings (from the vector) because the thread is also trying to assign the strings to another string.

This might indicate that the programmer was aware of the fact that the refence was not reliable and tries to copy it to a thread-local variable. Maybe it's just that there's a race condition involved and the strings don't even live long enough for the local copy to work.

IOW. It would appear that a thread is being started as such:

#include <future>
#include <algorithm>
#include <sstream>
#include <iterator>
#include <iostream>

using namespace std;

// sample: return the largest integer value or 42
int some_background_worker(vector<string> const& ref_data)
{
    if (ref_data.empty()) return 42;

    vector<int> values(ref_data.size());
    transform(begin(ref_data), end(ref_data), begin(values), 
            [](string const& s) { return stoi(s); });
    return *max_element(begin(values), end(values));
}

struct Parser
{
    future<int> parse(string const& msg) const
    {
        istringstream iss(msg);
        istream_iterator<string> f(iss), l;

        const vector<string> data { f, l };

        // now dispatch a thread to do the work
        return async(some_background_worker, ref(data));
    }
};

int main() // mimicks your "fifoThread" function
{
    // we use a simple vector to stub your fifo queue:
    const std::vector<string> fifoQueue { "1 3 9 -1 2", "32389 3102 -34 -888", "-42 -889", "" };

    vector<future<int>> results;

    {
        Parser parser;
        for(auto& msg : fifoQueue)
            results.push_back(parser.parse(msg));
    }

    // all parser data has been throroughly destroyed, before we might even
    // start the workers
    for(auto& fut: results)
        std::cout << "result of worker: " << fut.get() << "\n";
}

Here, the same race condition occurs, as a reference to "soon-to-be-stale" data is passed to the thread:

async(some_background_worker, ref(data));

It crashes with bad_alloc on my system (which is bogus, but perfectly fine Undefined Behaviour).

Like I said above, the std::string::assign calls might indicate the thread attempts to copy the data locally, like:

vector<string> local_copy(ref_data);

But that will still be a data race, because (without explicit synchronization) there is no guarantee that ref_data is valid even at that point.

---

Fix it by moving the data into the thread instead:

int some_background_worker(vector<string> data);

// and then, in parse(...):
    async(some_background_worker, std::move(data));

Now, on my system, it runs and prints

result of worker: 9
result of worker: 32389
result of worker: -42
result of worker: 42

fully as expected. See it running live at Coliru