Run Python3 without activating the virtual environment

问题

My objective is to run Python 3 code on the AWS Lambda Service, which currently only supports Python 2.7. These are the steps I have done.

Since I work on a Mac, setup a docker image similar to the AWS Lambda Linux instance.
Build Python3 from source on the docker image.
In the docker image create a virtual environment and copy it to my project.

AWS Lambda requires you to create a zip of the code and upload it to their service. For this prototype, I have a zip with three artifacts at the root

handler.py: This is a Python 2.7 file. The handler function in this file will be executed by the AWS Lambda Service when an event occurs (e.g. When a new file is created in a S3 bucket).

def handler(event, context):
    execution_uuid = uuid.uuid4()
    commands = '''
    source venv/bin/activate && venv/bin/python3.6 ./handler_python3.py --execution_uuid {ex_uuid}
    '''.format(ex_uuid=str(execution_uuid))
    p = Popen('/bin/bash', shell=True, stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True)
    stdout, stderr = p.communicate(commands)
    pprint(stdout)
    pprint(stderr)

handler_python3.py. This is the Python3 file called by the earlier handler.py file. Note the execution_uuid being read. I have taken out the code that uses it for brevity but I do need it and I am using argparse to extract it.

def read_execution_uuid():
    import argparse
    parser = argparse.ArgumentParser()
    parser.add_argument("--execution_uuid", required=True)
    args = parser.parse_args()

    return args.execution_uuid

def handler(event, context):
    import sys
    print(sys.path)

if __name__ == '__main__':
    execution_uuid = read_execution_uuid()
    handler(event, context)

venv folder. This is the virtual environment folder that I copied from the docker image.

When I run the AWS Lambda Service, I get the following error

Traceback (most recent call last):
  File "./handler_python3.py", line 38, in <module>
    execution_uuid = read_execution_uuid()
  File "./handler_python3.py", line 7, in read_execution_uuid
    import argparse
ModuleNotFoundError: No module named \'argparse\'

Notes:

If I remove the argparse code and the handler function in handler_python3.py executes, it shows the following values for sys.path

['/var/task', '/var/runtime', '/var/task/venv/lib/python36.zip', '/var/task/venv/lib/python3.6', '/var/task/venv/lib/python3.6/lib-dynload', '/usr/local/lib/python3.6', '/var/task/venv/lib/python3.6/site-packages']

Notes:

I can install argparse explicitly. But I'd like not to.
Note the source venv/bin/activate command in the python 2.7 file handler.py. That doesn't work on the Lambda instance though it works locally.

回答1:

Creating a virtual environment does not copy all the modules from the /usr/local/lib/python3.6 directory. I had to copy all the files there.

来源：https://stackoverflow.com/questions/42520389/run-python3-without-activating-the-virtual-environment

标签

python

virtualenv

aws-lambda