Making a very large Java array

Question

I\'m trying to find a counterexample to the Pólya Conjecture which will be somewhere in the 900 millions. I\'m using a very efficient algorithm that doesn\'t even require an

Answer 1

could you get by with 900 million bits? (maybe stored as a byte array).

Answer 2

You may want to extend the max size of the JVM Heap. You can do that with a command line option.

I believe it is -Xmx3600m (3600 megabytes)

Answer 3

Java arrays are indexed by int, so an array can't get larger than 2^31 (there are no unsigned ints). So, the maximum size of an array is 2147483648, which consumes (for a plain int[]) 8589934592 bytes (= 8GB).

Thus, the int-index is usually not a limitation, since you would run out of memory anyway.

In your algorithm, you should use a List (or a Map) as your data structure instead, and choose an implementation of List (or Map) that can grow beyond 2^31. This can get tricky, since the "usual" implementation ArrayList (and HashMap) uses arrays internally. You will have to implement a custom data structure; e.g. by using a 2-level array (a list/array). When you are at it, you can also try to pack the bits more tightly.

Answer 4

Use Tokyo Cabinet, Berkeley DB, or any other disk-based key-value store. They're faster than any conventional database but allow you to use the disk instead of memory.

Answer 5

If your algorithm allows it:

• Compute it in slices which fit into memory.

  You will have to redo the computation for each slice, but it will often be fast enough.

• Use an array of a smaller numeric type such as byte.

Answer 6

You can try splitting it up into multiple arrays.

for(int x = 0; x <= 1000000; x++){
    myFirstList.add(x);
}
for(int x = 1000001; x <= 2000000; x++){
    mySecondList.add(x);
}

then iterate over them.

for(int x: myFirstList){
    for(int y: myFirstList){
        //Remove multiples
    }
}
//repeat for second list