boost::asio async_read guarantee all bytes are read

Question

I have a server that receives a compressed string (compressed with zlib) from a client, and I was using async_receive from the boost::asio library to receive this string, it turns out however that there is no guarantee that all bytes will be received, so I now have to change it to async_read. The problem I face is that the size of the bytes received is variable, so I am not sure how to use async_read without knowing the number of bytes to be received. With the async_receive I just have a boost::array<char, 1024>, however this is a buffer that is not necessarily filled completely.

I wondered if anyone can suggest a solution where I can use async_read even though I do not know the number of bytes to be received in advance?

void tcp_connection::start(boost::shared_ptr<ResolverQueueHandler> queue_handler)
{
    if (!_queue_handler.get())
        _queue_handler = queue_handler;

    std::fill(buff.begin(), buff.end(), 0);

    //socket_.async_receive(boost::asio::buffer(buff), boost::bind(&tcp_connection::handle_read, shared_from_this(), boost::asio::placeholders::error));
    boost::asio::async_read(socket_, boost::asio::buffer(buff), boost::bind(&tcp_connection::handle_read, shared_from_this(), boost::asio::placeholders::error));
}

buff is a boost::array<char, 1024>

Answer 1

How were you expecting to do this using any other method?

There are a few general methods to sending data of variable sizes in an async manor:

1. By message - meaning that you have a header that defines the length of the expected message followed by a body which contains data of the specified length.
2. By stream - meaning that you have some marker (and this is very broad) method of knowing when you've gotten a complete packet.
3. By connection - each complete packet of data is sent in a single connection which is closed once the data is complete.

So can your data be parsed, or a length sent etc...

Answer 2

Use async_read_until and create your own match condition, or change your protocol to send a header including the number of bytes to expect in the compressed string.

Answer 3

A single IP packet is limited to an MTU size of ~1500 bytes, and yet still you can download gigabyte-large files from your favourite website, and watch megabyte-sized videos on YouTube.

You need to send a header indicating the actual size of the raw data, and then receive the data by pieces on smaller chunks until you finish receiving all the bytes.

For example, when you download a large file over HTTP, there is a field on the header indicating the size of the file: Content-Length:.