How do I load python modules which are not available in Sagemaker?

Question

I want to install spacy which is not available as part of the Sagemaker platform. How should can I pip install it?

Answer 1

Great answer from Raman. I wanted to add another way of specifying the required python modules in the training instance, in case someone is looking.

tf_estimator = TensorFlow(entry_point='tf-train.py', role='SageMakerRole',
                          training_steps=10000, evaluation_steps=100,
                          train_instance_count=1,
                          source_dir='./',
                          requirements_file='requirements.txt',
                          train_instance_type='ml.p2.xlarge')

source_dir and requirements_file both have to be defined for it to work. The path is wrt to the notebook instance. If requirements.txt is under the same directory as the notebook, then just use './'

Docs is here.