Fastest Way to Delete a Line from Large File in Python

前端 未结 9 604
轮回少年
轮回少年 2020-11-30 03:58

I am working with a very large (~11GB) text file on a Linux system. I am running it through a program which is checking the file for errors. Once an error is found, I need

9条回答
  •  自闭症患者
    2020-11-30 04:51

    I will provide two alternatives based on the look-up factor (line number or a search string):

    Line number

    def removeLine2(filename, lineNumber):
        with open(filename, 'r+') as outputFile:
            with open(filename, 'r') as inputFile:
    
                currentLineNumber = 0 
                while currentLineNumber < lineNumber:
                    inputFile.readline()
                    currentLineNumber += 1
    
                seekPosition = inputFile.tell()
                outputFile.seek(seekPosition, 0)
    
                inputFile.readline()
    
                currentLine = inputFile.readline()
                while currentLine:
                    outputFile.writelines(currentLine)
                    currentLine = inputFile.readline()
    
            outputFile.truncate()
    

    String

    def removeLine(filename, key):
        with open(filename, 'r+') as outputFile:
            with open(filename, 'r') as inputFile:
                seekPosition = 0 
                currentLine = inputFile.readline()
                while not currentLine.strip().startswith('"%s"' % key):
                    seekPosition = inputFile.tell()
                    currentLine = inputFile.readline()
    
                outputFile.seek(seekPosition, 0)
    
                currentLine = inputFile.readline()
                while currentLine:
                    outputFile.writelines(currentLine)
                    currentLine = inputFile.readline()
    
            outputFile.truncate()
    

提交回复
热议问题