Faster way of copying data in Java?

馋奶兔 提交于 2020-01-22 10:07:05

问题


I have been given a task of copying data from a server. I am using BufferedInputStream and output stream to copy the data and I am doing it byte by byte. Even though it is running but It is taking ages to copy the data as some of them are in 100's MBs, so definitely it is not gonna work. Can anyone suggest me any alternate of Byte by Byte copy so that my code can copy file that are in few Hundred MBs. Buffer is 2048.

Here is how my code look like:

static void copyFiles(SmbFile[] files, String parent) throws IOException {

  SmbFileInputStream input = null;
  FileOutputStream output = null;
  BufferedInputStream buf_input = null;
  try {
    for (SmbFile f : files) {
      System.out.println("Working on files :" + f.getName());
      if (f.isDirectory()) {

        File folderToBeCreated = new File(parent+f.getName());
        if (!folderToBeCreated.exists()) {
          folderToBeCreated.mkdir();
          System.out.println("Folder name " + parent
                + f.getName() + "has been created");
        } else {
          System.out.println("exists");

        }

        copyFiles(f.listFiles(), parent +  f.getName());
      } else {

        input = (SmbFileInputStream) f.getInputStream();

        buf_input = new BufferedInputStream(input, BUFFER);

        File t = new File(parent + f.getName());
        if (!t.exists()) {
          t.createNewFile();
        }
        output = new FileOutputStream(t);

        int c;

        int count;
        byte data[] = new byte[BUFFER];

        while ((count = buf_input.read(data, 0, BUFFER)) != -1) {
          output.write(data, 0, count);
        }
      }
    }
  } catch (IOException e) {
    e.printStackTrace();

  } finally {
    if (input != null) {
      input.close();
    }
    if (output != null) {
      output.close();
    }
  }
}

回答1:


Here is a link to an excellent post explaining how to use nio channels to make copies of streams. It introduces a helper method ChannelTools.fastChannelCopy that lets you copy streams like this:

final InputStream input = new FileInputStream(inputFile);
final OutputStream output = new FileOutputStream(outputFile);
final ReadableByteChannel inputChannel = Channels.newChannel(input);
final WriteableByteChannel outputChannel = Channels.newChannel(output);
ChannelTools.fastChannelCopy(inputChannel, outputChannel);
inputChannel.close();
outputChannel.close()



回答2:


Well since you're using a BufferedInputStream, you aren't reading byte by byte, but rather the size of the buffer. You could just try increasing the buffer size.




回答3:


Reading/writing byte-by-byte is definitely going to be slow, even though the actual reading/writing is done by chunks of the buffer size. One way to speed it up is to read/write by blocks. Have a look at read(byte[] b, int off, int len) method of BufferedInputStream. However it probably won't give you enough of the improvement.

What would be much better is to use nio package (new IO) to copy data using nio channels. Have a look at nio documentation for more info.




回答4:


I would suggest to use FileUtils from org.apache.commons.io. It has enough utility methods to perform file operations.

org.apache.commons.io.FileUtils API Here



来源:https://stackoverflow.com/questions/13748205/faster-way-of-copying-data-in-java

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!