The parent process fails with errno=12(Out of memory) when it tries to fork a child. The parent process runs on Linux 3.0 kernel - SLES 11. At the point of forking the child
A nicer solution on Linux would be to use vfork or posix_spawn (as it will try to use vfork if possible): vfork "creates new processes without copying the page tables of the parent process", so it will work even if your application uses more than 50% of RAM available.
Note that std::system and QProcess::execute also use fork under the hood, there is even a ticket about this problem in Qt framework: https://bugreports.qt.io/browse/QTBUG-17331