Input/output error while using google colab with google drive

Question

When I use google colab I get this error more than once and randomly Sometimes it works and sometimes not

OSError: [Errno 5] Input/output error

Answer 1

Another possible solution would be to save your files in a different (new folder) directory. I think @bob-smith's solution is one the best solutions for this problem, I am just showing a variation of the original solution which worked for me.

Answer 2

From the FAQ --

Google Drive operations can time out when the number of files or subfolders in a folder grows too large. If thousands of items are directly contained in the top-level "My Drive" folder then mounting the drive will likely time out. Repeated attempts may eventually succeed as failed attempts cache partial state locally before timing out. If you encounter this problem, try moving files and folders directly contained in "My Drive" into sub-folders. A similar problem can occur when reading from other folders after a successfuldrive.mount(). Accessing items in any folder containing many items can cause errors like OSError: [Errno 5] Input/output error (python 3) or IOError: [Errno 5] Input/output error (python 2). Again, you can fix this problem by moving directly contained items into sub-folders.

Answer 3

I ran into this error while using os.listdir on a google drive folder that had over 5.5k files in it, and a little window in the bottom left corner of my colab notebook popped up saying a timeout had occurred.

Because I have Colab Pro, I tried switching my runtime Hardware Accelerator to a GPU and Runtime Shape to High-Ram. This fixed the problem for me. It might have been one or both of those options together, not sure.

The problem with the top answer is that you might need some simple functionality in Colab (like os.listdir) in order to efficiently move files and create sub-folders to achieve reduced folder contents. If you can't even list what's in a folder without a timeout error occurring, you may just need to upgrade to Colab Pro to gain those advanced runtime options for a more powerful computing environment.