Check if the integer is a "probable prime." If it's not you know for sure that it's composite and you avoid the slow factorization.
if (!testNumber.isProbablePrime(5)) return false;
Also you only need to make trial divisions only up to the square root of testNumber. If K is composite you know that its least prime factor must be at most sqrt(K).