I'm developing form submission service with Jersey 2.0. The form includes several text fields and one file field. I need to extract file, file name, file media type and file content type and save them in object store.

@Path("upload")
@Consumes({MediaType.MULTIPART_FORM_DATA})
@Produces({MediaType.APPLICATION_XML, MediaType.APPLICATION_JSON})
public class UploadService {
    @POST
    public BlobDo uploadFile(FormDataMultiPart uploadedBody) {
        String accountSid = uploadedBody.getField("account-sid").getValue();
        String apiToken = uploadedBody.getField("api-token").getValue();
        String checksum = uploadedBody.getField("checksum").getValue();

        FormDataBodyPart bodyPart = uploadedBody.getField("file");
        MySwiftObject obj = new MySwiftObject(bodyPart.getValueAs(InputStream.class));
        obj.setName(bodyPart.getContentDisposition().getFileName());
        obj.setContentType(bodyPart.getMediaType().toString());
        obj.setContentDisposition(bodyPart.getContentDisposition().toString());
   ...
}

pom.xml

<jersey.version>2.17</jersey.version>

<dependency>
    <groupId>org.glassfish.jersey.containers</groupId>
    <artifactId>jersey-container-servlet-core</artifactId>
</dependency>
<dependency>
    <groupId>org.glassfish.jersey.media</groupId>
    <artifactId>jersey-media-json-jackson</artifactId>
</dependency>
<dependency>
    <groupId>org.glassfish.jersey.test-framework.providers</groupId>
    <artifactId>jersey-test-framework-provider-inmemory</artifactId>
</dependency>
<dependency>
    <groupId>org.glassfish.jersey.media</groupId>
    <artifactId>jersey-media-multipart</artifactId>
</dependency>
<dependency>
    <groupId>org.glassfish.jersey.core</groupId>
    <artifactId>jersey-server</artifactId>
</dependency>
<dependency>
    <groupId>org.glassfish.jersey.ext</groupId>
    <artifactId>jersey-bean-validation</artifactId>
</dependency>

form submission request

POST /nbs/v2/upload HTTP/1.1
Host: 127.0.0.1:8080
Cache-Control: no-cache
Postman-Token: a4c1d4e9-5f71-2321-3870-e9cac0524f8d
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryA2Z9pPMA7y3da8BG

------WebKitFormBoundaryA2Z9pPMA7y3da8BG
Content-Disposition: form-data; name="account-sid"

Q45Ppm5ukvdqjTQ6eW0O5ztTXipwnjKQx1p6cf+fbCQ=
------WebKitFormBoundaryA2Z9pPMA7y3da8BG
Content-Disposition: form-data; name="api-token"

6397cd691909fdc14cef67dbc1dc2dc3
------WebKitFormBoundaryA2Z9pPMA7y3da8BG
Content-Disposition: form-data; name="file"; filename="screen_4_100155.jpg"
Content-Type: image/jpeg

......Exif..MM.*.............................b...........j
------WebKitFormBoundaryA2Z9pPMA7y3da8BG
Content-Disposition: form-data; name="checksum"

6a3381b1d16bded4a3dfc325a8bb800e
------WebKitFormBoundaryA2Z9pPMA7y3da8BG

JVM heap size

-Xmx=1024mb

The problem

When uploading ~50MB file two temporary files with similar MD5 sums are created in the directory /tmp/tomcat7-tomcat7-tmp with name FileBackedOutputStream7949386530699987086.tmp and MIME8234229766850016150.tmp

Before upload is complete server throws exception

javax.servlet.ServletException: org.glassfish.jersey.server.ContainerException: java.lang.OutOfMemoryError: Java heap space
    org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:421)
    org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:386)
    org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:335)
    org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:222)

and deletes the file with name MIME8234229766850016150.tmp but leaves the other one. And undeleted FileBackedOutputStream....tmp files fill the whole space on hard drive.

What I did

Increase heap space to 7GB, but ~200MB files can't be uploaded.
Run job on server to delete old temporary files.
Created file with name jersey-multipart-config.properties and content

jersey.config.multipart.bufferThreshold = -1

The file MIME[random numbers].tmp is no longer created, but FileBackedOutputStream[random number].tmp still hangs on hard drive unless tomcat is restarted.

Question

How can Jersey handle large files (maybe 1GB) without leaving temporary files on my hard disk? Best case would be not to use hard drive at all and transfer small chunks through memory.
Why do I get heap overflow if input stream is backed with files?

Materials I read

The closes explanation I found so far. read
This guy has similar problem but on the client side. read
May contain solution, but couldn't understand the answer. read
Pretty close to my problem but can't work it out. read
bufferThreshold idea was taken from here. read

It seems that problem #1 was solved by adding following lines in my web.xml under the <servlet> tag

<multipart-config>
        <location>/tmp</location>
        <max-file-size>1000000000</max-file-size>
        <max-request-size>1500000000</max-request-size>
        <file-size-threshold>0</file-size-threshold>
</multipart-config>

and removed jersey-multipart-config.properties file.

Now I can upload files that are over 200Mb. No more temporary files are created.

But I still can't explain problem #2.

来源：https://stackoverflow.com/questions/31138727/jersey-service-file-upload-causes-outofmemoryerror

标签

file-upload