Include python module (dependencies) installation in your python script

Question

Is there a way to include/invoke python module(s) (dependencies) installation first, before running the actual/main script?

For example, in my main.py:

Answer 1

Is there a way to include/invoke python module(s) (dependencies) installation first, before running the actual/main script?

• A good way is to use setuptools and explicitly list them in install_requires.
• Since you are providing a main function, you also probably want to provide entry_points.

I already know the basics of setuptools. The problem is I may have to call the installation (setup.py) and the main script (main.py) separately.

That is usually not a problem. It is very common to first install everything with a requirements.txt file and pip install -r requirements.txt. Plus if you list dependencies you can then have reasonable expectations that it will be there when your function is called and not rely on try/except ImporError. It is a reasonable approach to expect required dependencies to be present and only use try/except for optional dependencies.

setuptools 101:

create a tree structure like this:

$ tree
.
├── mymodule
│   ├── __init__.py
│   └── script.py
└── setup.py

your code will go under mymodule; let's imagine some code that does a simple task:

# module/script.py    

def main():
    try:
        import requests
        print 'requests is present. kudos!'
    except ImportError:
        raise RuntimeError('how the heck did you install this?')

and here is a relevant setup:

# setup.py

from setuptools import setup
setup(
    name='mymodule',
    packages=['mymodule'],
    entry_points={
        'console_scripts' : [
            'mycommand = mymodule.script:main',
        ]
    },
    install_requires=[
        'requests',
    ]
)

This would make your main available as a command, and this would also take care of installing the dependencies you need (e.g requests)

~tmp damien$ virtualenv test && source test/bin/activate && pip install mymodule/
New python executable in test/bin/python
Installing setuptools, pip...done.
Unpacking ./mymodule
  Running setup.py (path:/var/folders/cs/nw44s66532x_rdln_cjbkmpm000lk_/T/pip-9uKQFC-build/setup.py) egg_info for package from file:///tmp/mymodule

Downloading/unpacking requests (from mymodule==0.0.0)
  Using download cache from /Users/damien/.pip_download_cache/https%3A%2F%2Fpypi.python.org%2Fpackages%2F2.7%2Fr%2Frequests%2Frequests-2.4.1-py2.py3-none-any.whl
Installing collected packages: requests, mymodule
  Running setup.py install for mymodule

    Installing mycommand script to /tmp/test/bin
Successfully installed requests mymodule
Cleaning up...
(test)~tmp damien$ mycommand 
requests is present. kudos!

more useful commands with argparse:

If you want to use argparse then...

# module/script.py

import argparse

def foobar(args):
    # ...

def main():
    parser = argparse.ArgumentParser()
    # parser.add_argument(...)
    args = parser.parse_args()
    foobar(args)

Answer 2

You should use the imp module. Here's a example:

import imp
import httplib2
import sys

try:
  import MultipartPostHandler
except ImportError:
  # Here you download 
  http = httplib2.Http()
  response, content = http.request('http://where_your_file_is.com/here')
  if response.status == 200:
    # Don't forget the right managment
    with open('MultipartPostHandler.py', 'w') as f:
     f.write(content)
    file, pathname, description = imp.find_module('MultipartPostHandler')
    MultipartPostHandler = imp.load_module('MultipartPostHandler', file, pathname, description)
  else:
    sys.exit('Unable to download the file')

For a full approach, use a queue:

download_list = []
try:
    import FirstModule
except ImportError:
    download_list.append('FirstModule')

try:
    import SecondModule
except ImportError:
    download_list.append('SecondModule')

if download_list:
    # Here do the routine to dowload, install and load_modules

# THe main routine
def main():
    the_response_is(42)

You can download binaries with open(file_content, 'wb')

I hope it help

BR

Answer 3

There's a few ways to do this. One way is to surround the import statement with a try...except ImportError block and then have some Python code that installs the package if the ImportError exception is raised, so something like:

try:
    import MultipartPostHandler
except ImportError:
    # code that installs MultipartPostHandler and then imports it

I don't think this approach is very clean. Plus if there are other unrelated importing issues, that won't be detected here. A better approach might be to have a bash script that checks to see if the module is installed:

pip freeze | grep MultipartPostHandler

and if not, installs the module:

pip install MultipartPostHandler

Then we can safely run the original Python code.

EDIT: Actually, I like FLORET's answer better. The imp module is exactly what you want.