Threadsafe and fault-tolerant file writes

Question

I have a long-running process which writes a lot of stuff in a file. The result should be everything or nothing, so I\'m writing to a temporary file and rename it to the rea

Answer 1

The with construct is useful for cleaning up on exit, but not for the commit/rollback system you want. A try/except/else block can be used for that.

You also should use a standard way for creating the temporary file name, for example with the tempfile module.

And remember to fsync before rename

Below is the full modified code:

import time, os, tempfile

def begin_file(filepath):
    (filedir, filename) = os.path.split(filepath)
    tmpfilepath = tempfile.mktemp(prefix=filename+'_', dir=filedir)
    return open(os.path.join(filedir, tmpfilepath), 'wb') 

def commit_file(f):
    tmppath = f.name
    (filedir, tmpname) = os.path.split(tmppath)
    origpath = os.path.join(filedir,tmpname.split('_')[0])

    os.fsync(f.fileno())
    f.close()

    if os.path.exists(origpath):
        os.unlink(origpath)
    os.rename(tmppath, origpath)

def rollback_file(f):
    tmppath = f.name
    f.close()
    os.unlink(tmppath)


fp = begin_file('whatever')
try:
    fp.write('stuff')
except:
    rollback_file(fp)
    raise
else:
    commit_file(fp)