Ensuring that my program is not doing a concurrent file write

后端 未结 2 988
面向向阳花
面向向阳花 2020-12-11 09:46

I am writing a script that is required to perform safe-writes to any given file i.e. append a file if no other process is known to be writing into it. My understanding of th

相关标签:
2条回答
  • 2020-12-11 10:09

    Use the lockfile module as shown in Locking a file in Python

    0 讨论(0)
  • 2020-12-11 10:12

    Inspired from a solution described for concurrency checks, I came up with the following snippet of code. It works if one is able to appropriately predict the frequency at which the file in question is written. The solution is through the use of file-modification times.

    import os
    import time
    
    '''Find if a file was modified in the last x seconds given by writeFrequency.'''
    def isFileBeingWrittenInto(filename, 
                           writeFrequency = 180, overheadTimePercentage = 20):
    
        overhead = 1+float(overheadTimePercentage)/100 # Add some buffer time
        maxWriteFrequency = writeFrequency * overhead
        modifiedTimeStart = os.stat(filename).st_mtime # Time file last modified
        time.sleep(writeFrequency)                     # wait writeFrequency # of secs
        modifiedTimeEnd = os.stat(filename).st_mtime   # File modification time again
        if 0 < (modifiedTimeEnd - modifiedTimeStart) <= maxWriteFrequency:
            return True
        else:
            return False
    
    if not isFileBeingWrittenInto('fileForSafeWrites.txt'):
        handle = open('fileForSafeWrites.txt', 'a')
        handle.write("Text written safely when no one else is writing to the file")
        handle.close()
    

    This does not do true concurrency checks but can be combined with a variety of other methods for practical purposes to safely write into a file without having to worry about garbled text. Hope it helps the next person searching for a way to do this.

    EDIT UPDATE:

    Upon further testing, I encountered a high-frequency write process that required the conditional logic to be modified from

    if 0 < (modifiedTimeEnd - modifiedTimeStart) < maxWriteFrequency 
    

    to

    if 0 < (modifiedTimeEnd - modifiedTimeStart) <= maxWriteFrequency 
    

    That makes a better answer, in theory and in practice.

    0 讨论(0)
提交回复
热议问题