AWS Lambda not importing LXML

Question

I am trying to use the LXML module within AWS Lambda and having no luck. I downloaded LXML using the following command:

pip install lxml -t folder
         


        

Answer 1

I faced the same issue.

The link posted by Raphaël Braud was helpful and so was this one: https://nervous.io/python/aws/lambda/2016/02/17/scipy-pandas-lambda/

Using the two links I was able to successfully import lxml and other required packages. Here are the steps I followed:

• Launch an ec2 machine with Amazon Linux ami

• Run the following script to accumulate dependencies:

  set -e -o pipefail
  sudo yum -y upgrade
  sudo yum -y install gcc python-devel libxml2-devel libxslt-devel
  
  virtualenv ~/env && cd ~/env && source bin/activate
  pip install lxml
  for dir in lib64/python2.7/site-packages \
       lib/python2.7/site-packages
  do
  if [ -d $dir ] ; then
     pushd $dir; zip -r ~/deps.zip .; popd
  fi
  done  
  mkdir -p local/lib
  cp /usr/lib64/ #list of required .so files
  local/lib/
  zip -r ~/deps.zip local/lib
  

• Create handler and worker files as specified in the link. Sample file contents:

handler.py

import os
import subprocess


libdir = os.path.join(os.getcwd(), 'local', 'lib')

def handler(event, context):
    command = 'LD_LIBRARY_PATH={} python worker.py '.format(libdir)
    output = subprocess.check_output(command, shell=True)

    print output

    return

worker.py:

import lxml

def sample_function( input_string = None):
    return "lxml import successful!"

if __name__ == "__main__":
    result = sample_function()
    print result

• Add handler and worker to zip file.

Here is how the structure of the zip file looks after the above steps:

deps 
├── handler.py
├── worker.py 
├── local
│   └── lib
│       ├── libanl.so
│       ├── libBrokenLocale.so
|       ....
├── lxml
│   ├── builder.py
│   ├── builder.pyc
|       ....
├── 

• Make sure you specify the correct handler name while creating the lambda function. In the above example, it would be- "handler.handler"

Hope this helps!