How can I analyze Python code to identify problematic areas?

Question

I have a large source repository split across multiple projects. I would like to produce a report about the health of the source code, identifying problem areas that need to be addressed.

Specifically, I'd like to call out routines with a high cyclomatic complexity, identify repetition, and perhaps run some lint-like static analysis to spot suspicious (and thus likely erroneous) constructs.

How might I go about constructing such a report?

Answer 1

For measuring cyclomatic complexity, there's a nice tool available at traceback.org. The page also gives a good overview of how to interpret the results.

+1 for pylint. It is great at verifying adherence to coding standards (be it PEP8 or your own organization's variant), which can in the end help to reduce cyclomatic complexity.

Answer 2

For cyclomatic complexity you can use radon: https://github.com/rubik/radon

(Use pip to install it: pip install radon)

Additionally it also has these features:

• raw metrics (these include SLOC, comment lines, blank lines, &c.)
• Halstead metrics (all of them)
• Maintainability Index (the one used in Visual Studio)

Answer 3

For static analysis there is pylint and pychecker. Personally I use pylint as it seems to be more comprehensive than pychecker.

For cyclomatic complexity you can try this perl program, or this article which introduces a python program to do the same

Answer 4

Pycana works like charm when you need to understand a new project!

PyCAna (Python Code Analyzer) is a fancy name for a simple code analyzer for python that creates a class diagram after executing your code.

See how it works: http://pycana.sourceforge.net/

output:

Answer 5

Thanks to Pydev, you can integrate pylint in the Eclipse IDE really easily and get a code report each time you save a modified file.

Answer 6

Use flake8, which provides pep8, pyflakes, and cyclomatic complexity analysis in one tool

Answer 7

There is a tool called CloneDigger that helps you find similar code snippets.

Answer 8

For checking cyclomatic complexity, there is of course the mccabe package.

Installation:

$ pip install --upgrade mccabe

Usage:

$ python -m mccabe --min=6 path/to/myfile.py

Note the threshold of 6 above. Per this answer, scores >5 probably should be simplified.

Sample output with --min=3:

68:1: 'Fetcher.fetch' 3
48:1: 'Fetcher._read_dom_tag' 3
103:1: 'main' 3

It can optionally also be used via pylint-mccabe or pytest-mccabe, etc.