unix: can i write to the same file in parallel without missing entries?

↘锁芯ラ 提交于 2020-01-04 05:47:11

问题


I wrote a script that executes commands in parallel. I let them all write an entry to the same log file. It does not matter if the order is wrong or entries are interleaved, but i noticed that some entries are missing. I should probably lock the file before writing, however, is it true that if multiple processes try to write to a file simultaneously, it will result in missing entries?


回答1:


Yes, if different processes independently open and write to the same file, it may result in overlapping writes and missing data. This happens because each process will get its own file pointer, that advances only by local writes.

Instead of locking, a better option might be to open the log file once in an ancestor of all worker processes, have it inherited across fork(), and used by them for logging. This means that there will be a single shared file pointer, that advances when any of the processes writes a new entry.




回答2:


In a script you should use ">> file" (double greater than) to append output to that file. The interpreter will open the destination in "append" mode. If your program also wants to append, follow the directives below:

Open a text file in "append" mode ("a+") and give preference to printing only full lines (don't do multiple 'print' followed by a final 'println', but print the entire line with a single 'println').

The fopen documentation states this:

DESCRIPTION
     The fopen() function opens the file whose  pathname  is  the
     string  pointed to by filename, and associates a stream with
     it.

     The argument mode points to a string beginning with  one  of
     the following sequences:

     r or rb             Open file for reading.


     w or wb             Truncate to  zero  length or create file
                         for writing.


     a or ab             Append; open or create file for  writing
                         at end-of-file.


     r+ or rb+ or r+b    Open file for update (reading and  writ-
                         ing).


     w+ or wb+ or w+b    Truncate to zero length or  create  file
                         for update.


     a+ or ab+ or a+b    Append; open or create file for  update,
                         writing at end-of-file.

The character b has no effect, but is allowed for ISO C standard conformance (see standards(5)). Opening a file with read mode (r as the first character in the mode argument) fails if the file does not exist or cannot be read.

Opening a file with append mode (a as the first character in the mode argument) causes all subsequent writes to the file to be forced to the then current end-of-file, regardless of intervening calls to fseek(3C). If two separate processes open the same file for append, each process may write freely to the file without fear of destroying output being written by the other. The output from the two processes will be intermixed in the file in the order in which it is written.

It is because of this intermixing that you want to give preference to using only 'println' (or its equivalent).



来源:https://stackoverflow.com/questions/15384331/unix-can-i-write-to-the-same-file-in-parallel-without-missing-entries

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!