Note: Version 2, below, uses the Sieve of Eratosthenes. There are several answers that helped with what I originally asked. I have chosen the Sieve of Era
As Paul Tomblin points out, there are better algorithms.
But keeping with what you have, and assuming an object per result is too big:
You are only ever appending to the array. So, use a relatively small int[] array. When it's full use append it to a List and create a replacement. At the end copy it into a correctly sized array.
Alternatively, guess the size of the int[] array. If it is too small, replace by an int[] with a size a fraction larger than the current array size. The performance overhead of this will remain proportional to the size. (This was discussed briefly in a recent stackoverflow podcast.)