How can I invoke a SageMaker model, trained with TensorFlow, using a csv file in the body of the call? [duplicate]

Question

This question already has an answer here:

• How do I load python modules which are not available in Sagemaker? 2 answers

I have deployed a TensorFlow model on AWS SageMaker, and I want to be able to invoke it using a csv file as the body of the call. The documentation says about creating a serving_input_function like the one below:

def serving_input_fn(hyperparameters):
  # Logic to the following:
  # 1. Defines placeholders that TensorFlow serving will feed with inference requests
  # 2. Preprocess input data
  # 3. Returns a tf.estimator.export.ServingInputReceiver or tf.estimator.export.TensorServingInputReceiver,
  # which packages the placeholders and the resulting feature Tensors together.

In step 2, where it says preprocess input data, how do I get a handle on input data to process them?

Answer 1

I had the same problem but I wanted to handle jpeg requests.

Once you have your model_data ready, you can deploy it with the following lines.

from sagemaker.tensorflow.model import TensorFlowModel
sagemaker_model = TensorFlowModel(
            model_data = 's3://path/to/model/model.tar.gz',
            role = role,
            framework_version = '1.12',
            entry_point = 'train.py',
            source_dir='my_src',
            env={'SAGEMAKER_REQUIREMENTS': 'requirements.txt'}
)

predictor = sagemaker_model.deploy(
    initial_instance_count=1,
    instance_type='ml.m4.xlarge', 
    endpoint_name='resnet-tensorflow-classifier'
)

Your notebook should have a my_src directory which contains a file train.py and a requirements.txt file. The train.py file should have a function input_fn defined. For me, that function handled image/jpeg content:

CSV_CONTENT_TYPE = 'text/csv'

# Deserialize the Invoke request body into an object we can perform prediction on
def input_fn(request_body, content_type=CSV_CONTENT_TYPE):
    # process an image uploaded to the endpoint
    if content_type == CSV_CONTENT_TYPE:
        ##handle input 
        return handled_input

    else: 
        raise errors.UnsupportedFormatError(content_type)

If yourtrain.py code imports some modules, you must supply requirements.txt defining those dependencies (that was the part I had trouble finding in the docs).

Hope this helps someone in the future.

Answer 2

You can preprocess input data by adding an input_fn which will be invoked every time u invoke and endpoint. It receives the input data and the content type of the data.

def input_fn(data, content_type):
    // do some data preprocessing.
    return preprocessed_data

This article explains it in more depth: https://docs.aws.amazon.com/sagemaker/latest/dg/tf-training-inference-code-template.html