Java — How to read an unknown number of bytes from an inputStream (socket/socketServer)?

Question

Looking to read in some bytes over a socket using an inputStream. The bytes sent by the server may be of variable quantity, and the client doesn\'t know in advance the lengt

Answer 1

This question is 7 years old, but i had a similiar problem, while making a NIO and OIO compatible system (Client and Server might be whatever they want, OIO or NIO).

This was quit the challenge, because of the blocking InputStreams.

I found a way, which makes it possible and i want to post it, to help people with similiar problems.

Reading a byte array of dynamic sice is done here with the DataInputStream, which kann be simply wrapped around the socketInputStream. Also, i do not want to introduce a specific communication protocoll (like first sending the size of bytes, that will be send), because i want to make this as vanilla as possible. First of, i have a simple utility Buffer class, which looks like this:

import java.util.ArrayList;
import java.util.List;

public class Buffer {

    private byte[] core;
    private int capacity;

    public Buffer(int size){
        this.capacity = size;
        clear();
    }

    public List list() {
        final List result = new ArrayList<>();
        for(byte b : core) {
            result.add(b);
        }

        return result;
    }

    public void reallocate(int capacity) {
        this.capacity = capacity;
    }

    public void teardown() {
        this.core = null;
    }

    public void clear() {
        core = new byte[capacity];
    }

    public byte[] array() {
        return core;
    }
}

This class only exists, because of the dumb way, byte <=> Byte autoboxing in Java works with this List. This is not realy needed at all in this example, but i did not want to leave something out of this explanation.

Next up, the 2 simple, core methods. In those, a StringBuilder is used as a "callback". It will be filled with the result which has been read and the amount of bytes read will be returned. This might be done different of course.

private int readNext(StringBuilder stringBuilder, Buffer buffer) throws IOException {
    // Attempt to read up to the buffers size
    int read = in.read(buffer.array());
    // If EOF is reached (-1 read)
    // we disconnect, because the
    // other end disconnected.
    if(read == -1) {
        disconnect();
        return -1;
    }
    // Add the read byte[] as
    // a String to the stringBuilder.
    stringBuilder.append(new String(buffer.array()).trim());
    buffer.clear();

    return read;
}

private Optional readBlocking() throws IOException {
    final Buffer buffer = new Buffer(256);
    final StringBuilder stringBuilder = new StringBuilder();
    // This call blocks. Therefor
    // if we continue past this point
    // we WILL have some sort of
    // result. This might be -1, which
    // means, EOF (disconnect.)
    if(readNext(stringBuilder, buffer) == -1) {
        return Optional.empty();
    }
    while(in.available() > 0) {
        buffer.reallocate(in.available());
        if(readNext(stringBuilder, buffer) == -1) {
            return Optional.empty();
        }
    }

    buffer.teardown();

    return Optional.of(stringBuilder.toString());
}

The first method readNext will fill the buffer, with byte[] from the DataInputStream and return the amount bytes read this way.

In the secon method, readBlocking, i utilized the blocking nature, not to worry about consumer-producer-problems. Simply readBlocking will block, untill a new byte-array is received. Before we call this blocking method, we allocate a Buffer-size. Note, i called reallocate after the first read (inside the while loop). This is not needed. You can safely delete this line and the code will still work. I did it, because of the uniqueness of my problem.

The 2 things, i did not explain in more detail are: 1. in (the DataInputStream and the only short varaible here, sorry for that) 2. disconnect (your disconnect routine)

All in all, you can now use it, this way:

// The in has to be an attribute, or an parameter to the readBlocking method
DataInputStream in = new DataInputStream(socket.getInputStream());
final Optional rawDataOptional = readBlocking();
rawDataOptional.ifPresent(string -> threadPool.execute(() -> handle(string)));

This will provide you with a way of reading byte arrays of any shape or form over a socket (or any InputStream realy). Hope this helps!