google-cloud-ml

Cannot run Google ML engine locally due to Tensorflow issues

由 你。 提交于 2019-12-02 22:11:49
问题 I'm trying to run the Google Cloud ML engine locally for debugging purposes by running the command gcloud ml-engine local predict --model-dir=fasttext_cloud/ --json-instances=debug_instance.json . However, I'm getting the error: ERROR: (gcloud.ml-engine.local.predict) Cannot import Tensorflow. This is strange as Tensorflow works fine on my machine. Even a simple example like python -c 'import tensorflow' has no issues whatsoever. 回答1: Is TensorFlow installed in a virtual environment or a non

Data Normalization with tensorflow tf-transform

由 不想你离开。 提交于 2019-12-02 19:48:39
I'm doing a neural network prediction with my own datasets using Tensorflow. The first I did was a model that works with a small dataset in my computer. After this, I changed the code a little bit in order to use Google Cloud ML-Engine with bigger datasets to realize in ML-Engine the train and the predictions. I am normalizing the features in the panda dataframe but this introduces skew and I get poor prediction results. What I would really like is use the library tf-transform to normalize the data in the graph. To do this, I would like to create a function preprocessing_fn and use the ' tft

Exit code non-zero and unable to see output logs

由 旧街凉风 提交于 2019-12-02 14:32:53
问题 How do I view stdout/stderr output logs for cloud ML? I've tried using gcloud beta logging read and also gcloud beta ml jobs stream-logs and nothing... all I see are the INFO level logs generated by the system i.e. "Tearing down TensorFlow". Also in the case where I have an error that shows the docker container exited with non zero code. It links me to a GUI page that shows the same stuff as gcloud beta ml jobs stream-logs. Nothing that shows me the actual output from the console of what my

Stackdriver logs not available for Cloud ML jobs since migration to V2

由 孤街醉人 提交于 2019-12-02 10:06:42
问题 Since migration to V2 logs from Cloud ML jobs are not accessible on the Stackdriver logging console anymore. The last log displayed is Waiting for Tensorflow to start. The job is executed and completed successfully, I just can't access outputs in the logs All Stackdriver APIs are enabled for the project. 回答1: There are no known issues with Cloud ML's Stackdriver logging. The fact that you see "Waiting for Tensorflow to start." indicates you are seeing log messages from Cloud ML. If logs from

Reading Input Data from GCS

由 拈花ヽ惹草 提交于 2019-12-02 08:04:19
问题 What is the suggest way of loading data from GCS? The sample code shows copying the data from GCS to the /tmp/ directory. If this is the suggest approach, how much data may be copied to /tmp/ ? 回答1: While you have that option, you shouldn't need to copy the data over to local disk. You should be able to reference training and evaluation data directly from GCS, by referencing your files/objects using their GCS URI -- eg. gs://bucket/path/to/file. You can use these paths where you'd normally

Exit code non-zero and unable to see output logs

由 北城余情 提交于 2019-12-02 07:20:59
How do I view stdout/stderr output logs for cloud ML? I've tried using gcloud beta logging read and also gcloud beta ml jobs stream-logs and nothing... all I see are the INFO level logs generated by the system i.e. "Tearing down TensorFlow". Also in the case where I have an error that shows the docker container exited with non zero code. It links me to a GUI page that shows the same stuff as gcloud beta ml jobs stream-logs. Nothing that shows me the actual output from the console of what my job produced... Help please?? It may be the case that the Cloud ML service account does not have

Stackdriver logs not available for Cloud ML jobs since migration to V2

由 主宰稳场 提交于 2019-12-02 04:42:25
Since migration to V2 logs from Cloud ML jobs are not accessible on the Stackdriver logging console anymore. The last log displayed is Waiting for Tensorflow to start. The job is executed and completed successfully, I just can't access outputs in the logs All Stackdriver APIs are enabled for the project. There are no known issues with Cloud ML's Stackdriver logging. The fact that you see "Waiting for Tensorflow to start." indicates you are seeing log messages from Cloud ML. If logs from your Python/TensorFlow program are missing that usually indicates Cloud ML hasn't been authorized to send

I get an error that gs:// path can not be read by service account

由 ≡放荡痞女 提交于 2019-12-02 04:17:15
问题 The GCS buckets I'm using for my ML jobs are encountering an access error. How do I make sure the service account has the required privileges? 回答1: I solved this by entering following: $ gcloud beta ml init-project Please see below for details. https://cloud.google.com/ml/docs/how-tos/getting-set-up 回答2: Make sure to allow the service account to read/write to your bucket. You will also need to change the default acl on your bucket so that new objects created in that bucket can be read by the

I get an error that gs:// path can not be read by service account

由 ﹥＞﹥吖頭↗ 提交于 2019-12-02 03:33:31
The GCS buckets I'm using for my ML jobs are encountering an access error. How do I make sure the service account has the required privileges? I solved this by entering following: $ gcloud beta ml init-project Please see below for details. https://cloud.google.com/ml/docs/how-tos/getting-set-up Make sure to allow the service account to read/write to your bucket. You will also need to change the default acl on your bucket so that new objects created in that bucket can be read by the service account. You can do this using these two gsutil commands: gsutil acl ch -u $SVCACCT:WRITE gs://$BUCKET/

import librosa in google cloud ml

由 ﹥＞﹥吖頭↗ 提交于 2019-12-02 02:21:30
I am running Google cloud ML and when I try to import librosa I get the error: ImportError: No module named _tkinter, please install the python-tk package I do have the have a setup.py File, an empty __init__.py file My full output from Google Cloud is the following: INFO 2017-02-10 12:45:53 -0800 unknown_task Validating job requirements... INFO 2017-02-10 12:45:53 -0800 unknown_task Job creation request has been successfully validated. INFO 2017-02-10 12:45:53 -0800 unknown_task Job urbanSond_guyknaan_20170210_124543 is queued. INFO 2017-02-10 12:46:02 -0800 unknown_task Waiting for job to be
订阅 google-cloud-ml