Is it safe to pipe the output of several parallel processes to one file using >>?

后端 未结 9 1289
一整个雨季
一整个雨季 2020-12-25 11:20

I\'m scraping data from the web, and I have several processes of my scraper running in parallel.

I want the output of each of these processes to end up in the same f

9条回答
  •  慢半拍i
    慢半拍i (楼主)
    2020-12-25 12:03

    No. It is not guaranteed that lines will remain intact. They can become intermingled.

    From searching based on liori's answer I found this:

    Write requests of {PIPE_BUF} bytes or less shall not be interleaved with data from other processes doing writes on the same pipe. Writes of greater than {PIPE_BUF} bytes may have data interleaved, on arbitrary boundaries, with writes by other processes, whether or not the O_NONBLOCK flag of the file status flags is set.

    So lines longer than {PIPE_BUF} bytes are not guaranteed to remain intact.

提交回复
热议问题