java.lang.OutOfMemoryError even though plenty of

Question

I am trying to read a 2.5GB txt file into my application. I am running Win7 x64 and have 43GB of mem available (out of 64GB). I tried playing around with -Xmx -XX:MaxParmS

Answer 1

You should look into the -Xmsn option for the java command.

It specifies the initial size of the memory allocation pool.

Edit: I see you've already done that.

Answer 2

You're trying to allocate an array that is too large. This is because you're trying to create a very long String. Since arrays are indexed by an integer, an array cannot have more than Integer.MAX_VALUE elements. Even if the your heap size is very large, you won't be able to allocate an array that has more than Integer.MAX_VALUE elements, simply because you cannot index its elements using an Integer. See Do Java arrays have a maximum size? for more details.

Answer 3

You can hold the data in string buffer data is List<String> in define interval and clear the StringBuffer.

Answer 4

You could create a new StringBuilder with a size, e.g.

    StringBuilder sb = new StringBuilder(Integer.MAX_VALUE);

The problem is that you're trying to read a file that's larger than what StringBuilder can have in its array. You have several options, e.g.:

1) Do you really need to read the entire file into memory at once? If so, you'll have to read it into several StringBuilders.

2) Process the file sequentially.

3) Read it into a compressed structure, and uncompress the parts you need when you need them.