Single thread pool vs one thread pool per task

Question

I want to use concurrency in Java to make requests to an online API, download and parse the response documents, and load the resulting data into a database.

Is it st

Answer 1

You have to consider which parts of the processing will benefit from parallelism. The online API communication will most likely be a candidate, since there will be sockets and network waits involved. Likewise with the DB interaction. Multithreaded parsing will probably only improve performance if there are multiple available CPU cores.

Splitting the entire process into 3 separate classes will definitely increase the cohesion, meaning each class will have less responsibilities, which is a good thing. On the other hand, making each of these classes a Runnable and having several queues will increase the complexity (possibly unecessarily) of the application.

I would suggest making 3 separate classes, but dont make them Runnable. Then make a Runnable that contains and orchestrates the 3 classes, that is one single thread pool. If you see that this doesnt seem to be fast enough (and after some profiling), try splitting the runnable into 2 thread pools: a download and parse, and a db access.

The point being, start simple and add complexity as needed.