Is python's shutil.copyfile() atomic?

问题

I'm writing a python script that copies a file using shutil.copyfile() on Linux. During the copying, other processes may be trying to read the file. Is the following sufficient to ensure that an external process doesn't get a corrupted view of the file?

os.unlink(dest)
shutil.copyfile(src, dest)

That is, is shutil.copyfile() atomic such that other processes cannot read the destination file until after the copy operation is complete?

回答1:

No, it seems to just loop, reading and writing 16KB at a time.

For an atomic copy operation, you should copy the file to a different location on the same filesystem, and then os.rename() it to the desired location (which is guaranteed to be atomic on Linux).

回答2:

No, shutil.copyfile is not atomic. This is part of the definition of shutil.copyfile:

def copyfile(src, dst, *, follow_symlinks=True):    
    ...
    with open(src, 'rb') as fsrc:
        with open(dst, 'wb') as fdst:
            copyfileobj(fsrc, fdst)

where copyfileobj is defined like this:

def copyfileobj(fsrc, fdst, length=16*1024):
    while 1:
        buf = fsrc.read(length)
        if not buf:
            break
        fdst.write(buf)

The thread calling copyfile could be stopped inside this while-loop at which point some other process could try to open the file to be read. It would get a corrupted view of the file.

来源：https://stackoverflow.com/questions/20873723/is-pythons-shutil-copyfile-atomic

标签

python

Linux

shutil