Check if my Python has all required packages

Question

I have a requirements.txt file with a list of packages that are required for my virtual environment. Is it possible to find out whether all the packages mentioned in the file are present. If some packages are missing, how to find out which are the missing packages?

Answer 1

The pythonic way of doing it is via the pkg_resources API. The requirements are written in a format understood by setuptools. E.g:

Werkzeug>=0.6.1
Flask
Django>=1.3

The example code:

import pkg_resources
from pkg_resources import DistributionNotFound, VersionConflict

# dependencies can be any iterable with strings, 
# e.g. file line-by-line iterator
dependencies = [
  'Werkzeug>=0.6.1',
  'Flask>=0.9',
]

# here, if a dependency is not met, a DistributionNotFound or VersionConflict
# exception is thrown. 
pkg_resources.require(dependencies)

Answer 2

You can run pip freeze to see what you have installed and compare it to your requirements.txt file.

If you want to install missing modules you can run pip install -r requirements.txt and that will install any missing modules and tell you at the end which ones were missing and installed.

Answer 3

Based on the answer by Zaur, assuming you indeed use a requirements.txt file, you may want a unit test, perhaps in tests/test_requirements.py, that confirms the availability of packages.

This approach uses a subtest to independently confirm each requirement. This is useful so that all failures are documented.

from pathlib import Path
import unittest

import pkg_resources


class TestRequirements(unittest.TestCase):
    def test_requirements(self):
        """Recursively confirm that requirements are available."""
        # Ref: https://stackoverflow.com/a/45474387/
        requirements = (Path(__file__).parents[1] / 'requirements.in').read_text().strip().split('\n')
        requirements = [r.strip() for r in requirements]
        requirements = [r for r in sorted(requirements) if r and not r.startswith('#')]
        for requirement in requirements:
            with self.subTest(requirement=requirement):
                pkg_resources.require(requirement)

Answer 4

If you're interested in doing this from the command line, pip check will list missing packages and any packages for which the wrong version is installed.

$ pip check
flake8-bugbear 19.8.0 requires flake8, which is not installed.
pyasn1-modules 0.2.6 has requirement pyasn1<0.5.0,>=0.4.6, but you have pyasn1 0.4.5.

If you want to focus on missing packages, ignoring version issues, you could filter the output with grep:

$ pip check | grep not\ installed

Answer 5

You can create a virtualenv with access to the system site packages and test check whether the package (or another dependencies) are installed or not. This way the packages are not really installed (if you just want to check). An example using virtualenv wrapper would be:

$ cat requirements.txt 
requests
simplejson

$ mkvirtualenv --system-site-packages test
Running virtualenv with interpreter /usr/bin/python2
New python executable in test/bin/python2
Also creating executable in test/bin/python
Installing setuptools, pip...done.

$ pip install -r requirements.txt
Downloading/unpacking requests (from -r requirements.txt (line 1))
  Downloading requests-2.10.0-py2.py3-none-any.whl (506kB): 506kB downloaded
Requirement already satisfied (use --upgrade to upgrade): simplejson in /usr/lib/python2.7/dist-packages (from -r requirements.txt (line 2))
Installing collected packages: requests
Successfully installed requests
Cleaning up...

$ pip install -r requirements.txt
Requirement already satisfied (use --upgrade to upgrade): requests in /home/yucer/.virtualenvs/test/lib/python2.7/site-packages (from -r requirements.txt (line 1))
Requirement already satisfied (use --upgrade to upgrade): simplejson in /usr/lib/python2.7/dist-packages (from -r requirements.txt (line 2))
Cleaning up...

$ deactivate

$ rmvirtualenv test
Removing test...

Answer 6

If requirements.txt is like :

django
oursql
sys
notexistingmodule

Then the following script will tell you which modules are missing :

#!/usr/bin/python3
fname = 'requirements.txt'
with open(fname, 'r', encoding='utf-8') as fhd:
    for line in fhd:
        try:
            exec("import " + line)
        except:
            print("[ERROR] Missing module:", line)

This would print :

[ERROR] Missing module: notexistingmodule