Access File through multiple threads

后端 未结 10 964
天涯浪人
天涯浪人 2021-01-31 10:52

I want to access a large file (file size may vary from 30 MB to 1 GB) through 10 threads and then process each line in the file and write them to another file through 10 threads

10条回答
  •  暗喜
    暗喜 (楼主)
    2021-01-31 11:33

    One of the possible ways will be to create a single thread that will read input file and put read lines into a blocking queue. Several threads will wait for data from this queue, process the data.

    Another possible solution may be to separate file into chunks and assign each chunk to a separate thread.

    To avoid blocking you can use asynchronous IO. You may also take a look at Proactor pattern from Pattern-Oriented Software Architecture Volume 2

提交回复
热议问题