How to ensure that we read all lines from boost::child process

Question

I saw the following code on boost::child documentation page where they explain how to read the output of a child process. http://www.boost.org/doc/libs/1_64_0/doc/h

Answer 1

I had the same issue... The best way to deal with this is by using async i/o.

Unfortunately, the boost documentation @ http://www.boost.org/doc/libs/master/doc/html/boost_process/extend.html#boost_process.extend.async was wrong... It makes it all look simple, but doesn't show that the buffers must be sized beforehand, and glosses over many details.

Here's my function, which send or receives one buffer in one go (no interaction like question/response0, I use stderr for error checking, since that's what was needed for my app, but you could catch the app's exit code by calling 'c.exit_code();`.

using tstring=basic_string;

void Run(
    const tstring& exeName;
    const tstring& args,
    const std::string& input,
    std::string& output,
    std::string& error
)
{
    using namespace boost;

    asio::io_service ios;

    std::vector vOut(128 << 10);
    auto outBuffer{ asio::buffer(vOut) };
    process::async_pipe pipeOut(ios);

    std::function onStdOut;
    onStdOut = [&](const system::error_code & ec, size_t n)
    {
        output.reserve(output.size() + n);
        output.insert(output.end(), vOut.begin(), vOut.begin() + n);
        if (!ec)
        {
            asio::async_read(pipeOut, outBuffer, onStdOut);
        }
    };

    std::vector vErr(128 << 10);
    auto errBuffer{ asio::buffer(vErr) };
    process::async_pipe pipeErr(ios);
    std::function onStdErr;
    onStdErr = [&](const system::error_code & ec, size_t n)
    {
        error.reserve(error.size() + n);
        error.insert(error.end(), vErr.begin(), vErr.begin() + n);
        if (!ec)
        {
            asio::async_read(pipeErr, errBuffer, onStdErr);
        }
    };

    auto inBuffer{ asio::buffer(input) };
    process::async_pipe pipeIn(ios);

    process::child c(
        exeName + _T(" ") + args, 
        process::std_out > pipeOut, 
        process::std_err > pipeErr, 
        process::std_in < pipeIn
    );


    asio::async_write(pipeIn, inBuffer, 
        [&](const system::error_code & ec, std::size_t n) 
        {
            pipeIn.async_close();
        });

    asio::async_read(pipeOut, outBuffer, onStdOut);
    asio::async_read(pipeErr, errBuffer, onStdErr);

    ios.run();
    c.wait();
}

The app I execute is a decoder/encoder, I send an entire file to process, and receive the results simultaneously this way. No file size limit.

IMPORTANT

There is a bug fix in boost 1.64, only affects Windows, apparently

in file boost\process\detail\windows\async_pipe.hpp: ref: https://github.com/klemens-morgenstern/boost-process/issues/90

line 79:

    ~async_pipe()
    {
//fix
        //if (_sink .native()  != ::boost::detail::winapi::INVALID_HANDLE_VALUE_)
        //    ::boost::detail::winapi::CloseHandle(_sink.native());
        //if (_source.native() != ::boost::detail::winapi::INVALID_HANDLE_VALUE_)
        //    ::boost::detail::winapi::CloseHandle(_source.native());
        boost::system::error_code ec;
        close(ec);
//fix
    }