Asynchronous HTTP Client for Java

Question

As a relative newbie in the Java world, I am finding many things frustratingly obtuse to accomplish that are relatively trivial in many other frameworks. A primary example i

Answer 1

If you haven't looked at it already, check out the Java 5 java.util.concurrent -- it makes multi-threaded apps much easier to develop. You can set up a ThreadPoolExecutor that manages, say, four Threads. You then feed the pool any number of tasks to complete. Each task is a Runnable. The ThreadPoolExecutor will queue up the Runnable tasks and feed them to available Threads in parallel. The Pool's afterExecute() method is called when each Runnable task completes.

I vividly remember writing a fetch thread pool for a web browser written in Java back in 1999, and it was a bear to get right. Last month I wrote a load tester for a web server. The tester has a ThreadPoolExecutor that has n threads, and the Runnable tasks I feed it each fetch a page using Apache HTTP Client. It took just an hour or two to get it working reasonably well. I think you'll like java.util.concurrent coupled with Apache HTTP Client, though it sounds like you'll need to do some customization for progress indication.

(Note that Apache HTTP Client does its own thread pooling, and the default configuration limits you to 20 threads max, and only two to each web server.)

Update: Here's the link to Apache HTTP Client. Be sure to read up on MultiThreadedHttpConnectionManager, it's what handles the connection pool, and it's not shown in the most basic example.