How to upload a Java OutputStream to AWS S3

问题

I create PDF docs in memory as OutputStreams. These should be uploaded to S3. My problem is that it's not possible to create a PutObjectRequest from an OutputStream directly (according to this thread in the AWS dev forum). I use aws-java-sdk-s3 v1.10.8 in a Dropwizard app.

The two workarounds I can see so far are:

1. Copy the OutputStream to an InputStream and accept that twice the amount of RAM is used.
2. Pipe the OutputStream to an InputStream and accept the overhead of an extra thread (see this answer)

If i don't find a better solution I'll go with #1, because it looks as if I could afford the extra memory more easily than threads/CPU in my setup.

Is there any other, possibly more efficient way to achive this that I have overlooked so far?

Edit: My OutputStreams are ByteArrayOutputStreams

回答1:

I solved this by subclassing ConvertibleOutputStream:

public class ConvertibleOutputStream extends ByteArrayOutputStream {
    //Craetes InputStream without actually copying the buffer and using up mem for that.
    public InputStream toInputStream(){
        return new ByteArrayInputStream(buf, 0, count);
    }
}

回答2:

What's the actual type of your OutputStream? Since it's an abstract class, there's no saying where the data actually goes (or if it even goes anywhere).

But let's assume that you're talking about a ByteArrayOutputStream since it at least keeps the data in memory (unlike many many others).

If you create a ByteArrayInputStream out of its buffer, there's no duplicated memory. That's the whole idea of streaming.

来源：https://stackoverflow.com/questions/31805518/how-to-upload-a-java-outputstream-to-aws-s3