I want to compress big text files with python (I am talking about >20Gb files). I am not any how an expert so I tried to gather the info I found and the following seems to w
Your script seems correct, but can be abbreviated:
from shutil import copyfileobj
with open('bigInputfile.txt', 'rb') as input:
with bz2.BZ2File('bigInputfile.txt.bz2', 'wb', compresslevel=9) as output:
copyfileobj(input, output)
Why are you calling the .close() methods? They are not needed as you use the with: statement.