How to download large files (like weights of a model) from Colaboratory?

Question

I have tried downloading small files from google Colaboratory. They are easily downloaded but whenever I try to download files which have a large sizes it shows an error? What is the way to download large files?

Answer 1

This is how I handle this issue:

from google.colab import auth
from googleapiclient.http import MediaFileUpload
from googleapiclient.discovery import build

auth.authenticate_user()

Then click on the link, authorize Google Drive and paste the code in the notebook.

drive_service = build('drive', 'v3')

def save_file_to_drive(name, path):
    file_metadata = {
      'name': name,
      'mimeType': 'application/octet-stream'
     }

     media = MediaFileUpload(path, 
                    mimetype='application/octet-stream',
                    resumable=True)

     created = drive_service.files().create(body=file_metadata,
                                   media_body=media,
                                   fields='id').execute()

     print('File ID: {}'.format(created.get('id')))

     return created

Then:

save_file_to_drive(destination_name, path_to_file)

This will save whatever files to your Google Drive, where you can download or sync them or whatever.

Answer 2

Google colab doesn't allow you to download large files using files.download(). But you can use one of the following methods to access it:

1. The easiest one is to use github to commit and push your files and then clone it to your local machine.
2. You can mount google-drive to your colab instance and write the files there.