Spring Batch - How to read one big file in multiple threads?

问题

Problem: Read file of size > 10 MB and load it in staging table using Spring Batch. How can we maintain state while reading a file, in order to restart the job if it fails?

As per the documentation the FileItemReader is not thread safe and if we try to make it thread safe, we end up loosing restartability. So basic questions are:

1. Is there a way to read the file in blocks and each thread knows which block it needs to read?
2. If we make the read synchronous, what changes are required to make the job restartable in this scenario?

If anyone has faced similar issues or have any analysis of how it performs would help us take decision.

Also, any pointers or sample codes are appreciated.

回答1:

Multithreading is only useful if your threads are doing different things at the same time. For example, you can have two threads running on separate CPUs. Or one thread can be waiting for a network message while the other is painting the screen.

But in your case, both threads would be waiting for the same IO from the same device, so there's no point using more than one.

See also this question Reading a file by multiple threads

来源：https://stackoverflow.com/questions/7275666/spring-batch-how-to-read-one-big-file-in-multiple-threads