google-colaboratory

Google colab TPU and reading from disc while traning

由 不羁的心 提交于 2019-12-08 07:24:39
问题 I have 100k pics, and it doesn't fit into ram, so I need read it from disc while training. dataset = tf.data.Dataset.from_tensor_slices(in_pics) dataset = dataset.map(extract_fn) def extract_fn(x): x = tf.read_file(x) x = tf.image.decode_jpeg(x, channels=3) x = tf.image.resize_images(x, [64, 64]) return x But then I try to train, I get this error File system scheme '[local]' not implemented (file: '/content/anime-faces/black_hair/danbooru_2629248_487b383a8a6e7cc0e004383300477d66.jpg') Can I

How to use Box 2 D on google colab

由 纵然是瞬间 提交于 2019-12-08 05:40:12
问题 I am currently trying to optimize OpenAIGym's BipedalWalker with neat. In order to use Bipedalwalker, it is necessary to install Box 2 D, but a problem arises. In order to install Box 2 d on colab, we did the following first. !apt-get install python-box2d > /dev/null !pip install gym[Box_2D] import gym env = gym.make("BipedalWalker-v2") However, this caused the following error /usr/local/lib/python3.6/dist-packages/gym/envs/box2d/lunar_lander.py in <module>() 2 import numpy as np 3 ----> 4

Colaboratory and Selenium: error message 'geckodriver' executable needs to be in PATH [duplicate]

由 人盡茶涼 提交于 2019-12-08 04:28:37
问题 This question already has answers here : Python 3.5 - “Geckodriver executable needs to be in PATH” (3 answers) WebDriverException: 'geckodriver' executable needs to be in PATH even though it is (1 answer) Closed last year . I try to use Selenium to run a script to download/scraping data from a "infinite" scroll instagram page for research pourpose. I use google colaboratory and this haven't a browser installed because operate like a server. It's my script import time from selenium import

Cupy OutOfMemoryError when trying to cupy.load larger dimension .npy files in memory map mode, but np.load works fine

由 家住魔仙堡 提交于 2019-12-08 03:31:16
问题 I'm trying to load some larger .npy files in cupy with memory mapped mode, but I keep running into OutOfMemoryError . I thought that since it's being opened in memory mapped mode, this operation shouldn't take much memory since a memory map doesn't actually load the whole array into memory. I can load these files with np.load just fine, this only seems to happen with cupy.load. My enviroment is Google Colab, with the Tesla K80 GPU. It has about 12 gigs CPU ram, 12 gigs GPU ram, and 350 gb

Checkpoints in Google Colab

由 拜拜、爱过 提交于 2019-12-08 02:47:25
How do I store my trained model on Google Colab and retrieve further on my local disk? Will checkpoints work? How do I store them and retrieve them after some time? Can you please mention code for that. It would be great. Google Colab instances are created when you open the notebook and are deleted later on so you can't access data on different runs. If you want to download the trained model to your local machine you can use: from google.colab import files files.download(<filename>) And similarly if you want to upload the model from your local machine you can do: from google.colab import files

Update keras.json on Google Colab

由 送分小仙女□ 提交于 2019-12-08 02:11:59
问题 I tried to update keras.json on Google Colabs, but its throwing UnsupportedOperation error Is there any other alternative to achieve this? 回答1: You're opening the file as readonly -- pass 'w' to the open call on line 9. (docs) 来源： https://stackoverflow.com/questions/50083075/update-keras-json-on-google-colab

Google colab changing directory

由 久未见 提交于 2019-12-08 01:55:26
问题 I can't change my directory in Google colab. when I type cd it gives output like this '/content' I tried to change the directory using import os os.chdir("drive") but it shows error No such file or directory: 'drive' . How to get rid of this? 回答1: Did you create a /drive directory? (By default, one does not exist. You'll need to mount your Drive using a recipe like this one. Otherwise, create a directory first. For example, !mkdir demo %cd demo !pwd will show /content/demo . 回答2: First, you

Saving Variable state in Colaboratory

由 纵然是瞬间 提交于 2019-12-08 00:01:54
问题 When I am running a Python Script in Colaboratory, it's running all previous code cell. Is there any way by which previous cell state/output can be saved and I can directly run next cell after returning to the notebook. 回答1: The outputs of Colab cells shown in your browser are stored in notebook JSON saved to Drive. Those will persist. If you want to save your Python variable state, you'll need to use something like pickle to save to a file and then save that file somewhere outside of the VM.

OSError: Input/Output Error - Google Colab

由 a 夏天 提交于 2019-12-07 15:56:27
Using h5py.File(path,'r') or (Keras) model.load_weights(path) on .h5 files yields the below error. I've trained models on Colab for months, never had this problem. The same files open just fine on my computer when downloaded from Drive. At first one notebook failed to open .h5 files - all others followed soon after. What's the error, and how to troubleshoot? Inability to save & load models renders Colab unusable - help is appreciated. UPDATE : What started as inability to open h5 files morphed into inability to import simple modules - which somehow restored ability to load some of the h5 files

How to download large files (like weights of a model) from Colaboratory?

由 ﹥＞﹥吖頭↗ 提交于 2019-12-07 13:13:39
问题 I have tried downloading small files from google Colaboratory. They are easily downloaded but whenever I try to download files which have a large sizes it shows an error? What is the way to download large files? 回答1: This is how I handle this issue: from google.colab import auth from googleapiclient.http import MediaFileUpload from googleapiclient.discovery import build auth.authenticate_user() Then click on the link, authorize Google Drive and paste the code in the notebook. drive_service =
订阅 google-colaboratory