Currently, the functioning prototype has everything it needs to work: A semi-transparent window for easy resizing on the place you want to record, multiple threads to manag
I suggest to use Xuggler for this. Here's an introduction.
I'd suggest just learning how to call external programs from your java program. Then you can just run that example program you linked to. Or if you want even more capability, use ffmpeg which will do this automatically from the command line as well:
ffmpeg -i screencapture%d.jpg -vcodec mpeg4 outfile.avi
ffmpeg allows you to customize a lot of things about the output video. Just check out the documentation for all the options.
The Java command Runtime.getRuntime().exec(...)
will let you run whatever command line you pass it. There are many examples of how to do this if you Google it.