A better algorithm might be the following for searching for primes (my java is rusty, so some tweaking will probably be needed to get it to compile).
if (number % 2)
factors.append(2)
if (number % 3)
factors.append(3)
for(int n = 0; n < sqrt(number)/6; n++)
{
if (number % (6 * n + 1))
factors.append(6 * n + 1);
if (number % (6 * n - 1))
factors.append(6 * n - 1);
}