The following code can read and write back to s3 on the fly following the the discussion on here:
from smart_open import open import os bucket_dir = "s3