Best way to Pipe InputStream to OutputStream [duplicate]

问题

I was to trying to find the best way to pipe the InputStream to OutputStream. I don't have an option to use any other libraries like Apache IO. Here is the snippet and output.

import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.nio.channels.FileChannel;

public class Pipe {
    public static void main(String[] args) throws Exception {

        for(PipeTestCase testCase : testCases) {
            System.out.println(testCase.getApproach());
            InputStream is = new FileInputStream("D:\\in\\lft_.txt");
            OutputStream os = new FileOutputStream("D:\\in\\out.txt");

            long start = System.currentTimeMillis();            
            testCase.pipe(is, os);
            long end = System.currentTimeMillis();

            System.out.println("Execution Time = " + (end - start) + " millis");
            System.out.println("============================================");

            is.close();
            os.close();
        }

    }

    private static PipeTestCase[] testCases = {

        new PipeTestCase("Fixed Buffer Read") {         
            @Override
            public void pipe(InputStream is, OutputStream os) throws IOException {
                byte[] buffer = new byte[1024];
                while(is.read(buffer) > -1) {
                    os.write(buffer);   
                }
            }
        },

        new PipeTestCase("dynamic Buffer Read") {           
            @Override
            public void pipe(InputStream is, OutputStream os) throws IOException {
                byte[] buffer = new byte[is.available()];
                while(is.read(buffer) > -1) {
                    os.write(buffer);   
                    buffer = new byte[is.available() + 1];
                }
            }
        },

        new PipeTestCase("Byte Read") {         
            @Override
            public void pipe(InputStream is, OutputStream os) throws IOException {
                int c; 
                while((c = is.read()) > -1) {
                    os.write(c);    
                }
            }
        }, 

        new PipeTestCase("NIO Read") {          
            @Override
            public void pipe(InputStream is, OutputStream os) throws IOException {
                FileChannel source      = ((FileInputStream) is).getChannel(); 
                FileChannel destnation  = ((FileOutputStream) os).getChannel();
                destnation.transferFrom(source, 0, source.size());
            }
        }, 

    };
}


abstract class PipeTestCase {
    private String approach; 
    public PipeTestCase( final String approach) {
        this.approach = approach;           
    }

    public String getApproach() {
        return approach;
    }

    public abstract void pipe(InputStream is, OutputStream os) throws IOException;
}

Output (~4MB input file) :

Fixed Buffer Read
Execution Time = 71 millis
============================================
dynamic Buffer Read
Execution Time = 167 millis
============================================
Byte Read
Execution Time = 29124 millis
============================================
NIO Read
Execution Time = 125 millis
============================================

'Dynamic Buffer Read' uses available() method. But it is not reliable as per java docs

It is never correct to use the return value of this method to allocate a buffer intended to hold all data in this stream.

'Byte Read' seems to be very slow.

So 'Fixed Buffer Read' is the best option for pipe? Any thoughts?

回答1:

I would say a fixed buffer size is the best/easiest to understand. However there are a few problems.

You're writing the entire buffer to the output stream each time. For the final block the read may have read < 1024 bytes so you need to take this into account when doing the write (basically only write number of bytes returned by read()
In the dynamic buffer case you use available(). This is not a terribly reliable API call. I'm not sure in this case inside a loop whether it will be ok, but I wouldn't be suprised if it was implemented sub-optimally in some implementations of InputStream.
The last case you are casting to FileInputStream. If you intend for this to be general purpose then you can't use this approach.

回答2:

I came across this, and the final read can cause problems.

SUGGESTED CHANGE:

public void pipe(InputStream is, OutputStream os) throws IOException {
  int n;
  byte[] buffer = new byte[1024];
  while((n = is.read(buffer)) > -1) {
    os.write(buffer, 0, n);   // Don't allow any extra bytes to creep in, final write
  }
 os.close ();

I also agree that 16384 is probably a better fixed buffer size than 1024.

IMHO...

回答3:

For people looking for an one-liner, here's a solution from apache commons:

IOUtils.copy(inputStream, outputStream);

Documentation here. There are multiple copy methods with different parameters. It is also possible to specify the buffer size.

回答4:

java.io contains PipedInputStream and PipedOutputStream

PipedInputStream input = new PipedInputStream();
PipedOutputStream output = new PipedOutputStream (input);

write to input and it will be visible in output as an Outputstream. Things can work the other way around as well

来源：https://stackoverflow.com/questions/11715979/best-way-to-pipe-inputstream-to-outputstream

标签

java

pipe

java-io