amazon-sagemaker | 易学教程

Sagemaker LDA topic model - how to access the params of the trained model? Also is there a simple way to capture coherence

阅读更多关于 Sagemaker LDA topic model - how to access the params of the trained model? Also is there a simple way to capture coherence

I'm new to Sagemaker and am running some tests to measure the performance of NTM and LDA on AWS compared with LDA mallet and native Gensim LDA model. I'm wanting to inspect the trained models on Sagemaker and look at stuff like what words have the highest contribution for each topic. And also to get a measure of model coherence. I have been able to successfully get what words have the highest contribution for each topic for NTM on Sagemaker by downloading the output file untarring it and unzipping to expose 3 files params, symbol.json and meta.json. However, when I try to do the same process

AWS Sagemaker | how to train text data | For ticket classification

阅读更多关于 AWS Sagemaker | how to train text data | For ticket classification

问题 I am new to Sagemaker and not sure how to classify the text input in AWS sagemaker, Suppose I have a Dataframe having two fields like 'Ticket' and 'Category', Both are text input, Now I want to split it test and training set and upload in Sagemaker training model. X_train, X_test, y_train, y_test = model_selection.train_test_split(fewRecords['Ticket'],fewRecords['Category']) Now as I want to perform TD-IDF feature extraction and then convert it to numeric value, so performing this operation

Invoke aws sagemaker endpoint

阅读更多关于 Invoke aws sagemaker endpoint

问题 I have some data in S3 and I want to create a lambda function to predict the output with my deployed aws sagemaker endpoint then I put the outputs in S3 again. Is it necessary in this case to create an api gateway like decribed in this link ? and in the lambda function what I have to put. I expect to put (where to find the data, how to invoke the endpoint, where to put the data) import boto3 import io import json import csv import os client = boto3.client('s3') #low-level functional API

Amazon SageMake throwing error Building your own algorithm container execution time?

阅读更多关于 Amazon SageMake throwing error Building your own algorithm container execution time?

问题 I am trying to run my own algorithm container in amazon sagemaker,at the time of deployment time ,I am getting error like below. predictor = tree.deploy(1, 'ml.m4.xlarge', serializer=csv_serializer) ValueError: Error hosting endpoint decision-trees-sample-2018-03-01-09-59-06-832: Failed Reason: The primary container for production variant AllTraffic did not pass the ping health check. then I run same line of code this time i am getting below error. predictor = tree.deploy(1, 'ml.m4.xlarge',

Sagemaker LDA topic model - how to access the params of the trained model? Also is there a simple way to capture coherence

阅读更多关于 Sagemaker LDA topic model - how to access the params of the trained model? Also is there a simple way to capture coherence

问题 I'm new to Sagemaker and am running some tests to measure the performance of NTM and LDA on AWS compared with LDA mallet and native Gensim LDA model. I'm wanting to inspect the trained models on Sagemaker and look at stuff like what words have the highest contribution for each topic. And also to get a measure of model coherence. I have been able to successfully get what words have the highest contribution for each topic for NTM on Sagemaker by downloading the output file untarring it and

AWS Sagemaker custom user algorithms: how to take advantage of extra instances

阅读更多关于 AWS Sagemaker custom user algorithms: how to take advantage of extra instances

问题 This is a fundamental AWS Sagemaker question. When I run training with one of Sagemaker's built in algorithms I am able to take advantage of the massive speedup from distributing the job to many instances by increasing the instance_count argument of the training algorithm. However, when I package my own custom algorithm then increasing the instance count seems to just duplicate the training on every instance, leading to no speedup. I suspect that when I am packaging my own algorithm there is

SageMaker fails when using Multi-GPU with keras.utils.multi_gpu_model

阅读更多关于 SageMaker fails when using Multi-GPU with keras.utils.multi_gpu_model

问题 Running AWS SageMaker with a custom model, the TrainingJob fails with an Algorithm Error when using Keras plus a Tensorflow backend in multi-gpu configuration: from keras.utils import multi_gpu_model parallel_model = multi_gpu_model(model, gpus=K) parallel_model.compile(loss='categorical_crossentropy', optimizer='rmsprop') parallel_model.fit(x, y, epochs=20, batch_size=256) This simple parallel model loading will fail. There is no further error or exception from CloudWatch logging. This

AWS Sagemaker custom user algorithms: how to take advantage of extra instances

阅读更多关于 AWS Sagemaker custom user algorithms: how to take advantage of extra instances

This is a fundamental AWS Sagemaker question. When I run training with one of Sagemaker's built in algorithms I am able to take advantage of the massive speedup from distributing the job to many instances by increasing the instance_count argument of the training algorithm. However, when I package my own custom algorithm then increasing the instance count seems to just duplicate the training on every instance, leading to no speedup. I suspect that when I am packaging my own algorithm there is something special I need to do to control how it handles the training differently for a particular

upload data to S3 with sagemaker

阅读更多关于 upload data to S3 with sagemaker

I have a problem with SageMaker when I try to upload Data into S3 bucket . I get this error : NameError Traceback (most recent call last) <ipython-input-26-d21b1cb0fcab> in <module>() 19 download('http://data.mxnet.io/data/caltech-256/caltech-256-60-train.rec') 20 ---> 21 upload_to_s3('train', 'caltech-256-60-train.rec') <ipython-input-26-d21b1cb0fcab> in upload_to_s3(channel, file) 13 data = open(file, "rb") 14 key = channel + '/' + file ---> 15 s3.Bucket(bucket).put_object(Key=key, Body=data) 16 17 NameError: name 'bucket' is not defined Here is the script: import os import urllib.request

SageMaker fails when using Multi-GPU with keras.utils.multi_gpu_model

阅读更多关于 SageMaker fails when using Multi-GPU with keras.utils.multi_gpu_model

Running AWS SageMaker with a custom model, the TrainingJob fails with an Algorithm Error when using Keras plus a Tensorflow backend in multi-gpu configuration: from keras.utils import multi_gpu_model parallel_model = multi_gpu_model(model, gpus=K) parallel_model.compile(loss='categorical_crossentropy', optimizer='rmsprop') parallel_model.fit(x, y, epochs=20, batch_size=256) This simple parallel model loading will fail. There is no further error or exception from CloudWatch logging. This configuration works properly on local machine with 2x NVIDIA GTX 1080, same Keras Tensorflow backend.