Process file in multithread in Node.js

问题

I need to read a file from online url and process it in multithread and write it in another output . https://www.w3.org/TR/PNG/iso_8859-1.txt Solution I tried : If cluster is master I forked new thread based on cpu count .and outside for loop read the file. I have to make sure each thread is given certain lines to print in output file as same line should not be repeated.

I am not sure if this is correct way to do it. Please share your answers or suggestion.

回答1:

Could you elaborate why you think you should process something in a different thread?

Please check my detailed reply here regarding a similar question: How to truly make a time consuming request async in node

To summarize:

• Node.JS is executing your javascript code in one thread only (mainthread)
• Built-in modules like fs, http, net etc. make use of libuv worker threads to process I/O in a background thread
• Thirdparty modules might do the same, it depends on their implementation
• If you want your code to be executed in a background thread, you have 2 options:
  • Write a C-wrapper module which uses libuv worker threads to do the stuff
  • Use WorkerThreads in your javascript code which were introduced in Node.JS 10 (Docs), but please note that the status of this module is experimental and not ready for production

Generally speaking, I highly doubt you want to use threads on your own. Your question indicates you want to download a file from a webserver. The available modules (fs, request, http, ...) already make use of background threads to do the I/O, so it is unclear to me where on top of that you would need background processing.

Please clarify more, or think about sticking to the available modules in case you don't need "more multithreading".

来源：https://stackoverflow.com/questions/54801751/process-file-in-multithread-in-node-js