General-purpose computing on graphics processing units (GPGPU) is a very attractive concept to harness the power of the GPU for any kind of computing.
I\'d love to
Well CUDA is a modification of C, to write CUDA kernel you have to code in C, and then compile to executable form with nvidia's CUDA compiler. Produced native code could then be linked with Java using JNI. So technically you can't write kernel code from Java. There is JCUDA http://www.jcuda.de/jcuda/JCuda.html, it provides you with cuda's apis for general memory/device menagement and some Java methods that are implemented in CUDA and JNI wrapped (FFT, some linear algebra methods.. etc etc..).
On the other hand OpenCL is just an API. OpenCL kernels are plain strings passed to the API so using OpenCL from Java you should be able to specify your own kernels. OpenCL binding for java can be found here http://www.jocl.org/.