Java TCP socket: data transfer is slow

Question

I set up a server with a ServerSocket, connect to it with a client machine. They\'re directly networked through a switch and the ping time is <1ms.

Now, I try to

Answer 1

How is your heap size set? I had a similar problem recently with the socket transfer of large amounts of data and just by looking at JConsole I realized that the application was spending most of its time doing full GCs.

Try -Xmx1g

Answer 2

Since I cannot yet comment on this site, I must write answer to @Erik here.

The problem is that DataOutputStream doesn't buffer. The whole Stream-thing in Java is based on decorators design pattern. So you could write

DataOutputStream out = new DataOutputStream(new BufferedOutputStream(socket.getOutputStream()));

It will wrap the original stream in a BufferedOutputStream which is more efficient, which is then wrapped into a DataOutputStream which offers additional nice features like writeInt(), writeLong() and so on.

Answer 3

You do not want to write single bytes when you are transferring large amounts of data.

import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.ServerSocket;
import java.net.Socket;

public class Transfer {

    public static void main(String[] args) {
        final String largeFile = "/home/dr/test.dat"; // REPLACE
        final int BUFFER_SIZE = 65536;
        new Thread(new Runnable() {
            public void run() {
                try {
                    ServerSocket serverSocket = new ServerSocket(12345);
                    Socket clientSocket = serverSocket.accept();
                    long startTime = System.currentTimeMillis();
                    byte[] buffer = new byte[BUFFER_SIZE];
                    int read;
                    int totalRead = 0;
                    InputStream clientInputStream = clientSocket.getInputStream();
                    while ((read = clientInputStream.read(buffer)) != -1) {
                        totalRead += read;
                    }
                    long endTime = System.currentTimeMillis();
                    System.out.println(totalRead + " bytes read in " + (endTime - startTime) + " ms.");
                } catch (IOException e) {
                }
            }
        }).start();
        new Thread(new Runnable() {
            public void run() {
                try {
                    Thread.sleep(1000);
                    Socket socket = new Socket("localhost", 12345);
                    FileInputStream fileInputStream = new FileInputStream(largeFile);
                    OutputStream socketOutputStream = socket.getOutputStream();
                    long startTime = System.currentTimeMillis();
                    byte[] buffer = new byte[BUFFER_SIZE];
                    int read;
                    int readTotal = 0;
                    while ((read = fileInputStream.read(buffer)) != -1) {
                        socketOutputStream.write(buffer, 0, read);
                        readTotal += read;
                    }
                    socketOutputStream.close();
                    fileInputStream.close();
                    socket.close();
                    long endTime = System.currentTimeMillis();
                    System.out.println(readTotal + " bytes written in " + (endTime - startTime) + " ms.");
                } catch (Exception e) {
                }
            }
        }).start();
    }
}

This copies 1 GiB of data in short over 19 seconds on my machine. The key here is using the InputStream.read and OutputStream.write methods that accept a byte array as parameter. The size of the buffer is not really important, it just should be a bit larger than, say, 5. Experiment with BUFFER_SIZE above to see how it effects the speed but also keep in mind that it probably is different for every machine you are running this program on. 64 KiB seem to be a good compromise.

Answer 4

How are you implementing the receiving end? Please post your receiving code as well.

Since TCP is a reliable protocol, it will take steps to make sure the client is able to receive all of the data sent by the sender. This means that if your client cannot get the data out of the data receive buffer in time, then the sending side will simply stop sending more data until the client has a chance to read all the bytes in the receiving buffer.

If your receiving side is reading data one byte at a time, then your sender probably will spend a lot of time waiting for the receiving buffer to clear, hence the long transfer times. I'll suggest changing your receiving code to reading as many bytes as possible in each read operation . See if that will solve your problem.

Answer 5

@Erik: using DataXxxputStream is not the problem here. Problem is you were sending data in too small chunks. Using a buffer solved your problem because even you would write bit by bit the buffer would solve the problem. Bombe's solution is much nicer, generic and faster.

Answer 6

USe Byte buffer for sending the data