Java is definitely slow, especially for quantitative work.
I use a combination of R, Python and C/C++ with optimised multithreaded ATLAS libraries. In each of these languages I can matrix multiply a 3000 by 3000 matrix of doubles with itself in around 4 seconds. Using Colt and Parallel Colt in Java, the same operation take 185 seconds! Astonishing despite these java libraries being parallel in nature.
So all in all, pure Java is unsuitable for quantitative work. Jblas seems to be the best linear algebra library for Java as it uses ATLAS.
My machine is an HP Core 2 Duo with 3 GB of RAM. I use 64-bit Ubuntu 10.04 (Lucid Lynx).