cxfreeze missing distutils module inside virtualenv

Question

When running a cxfreeze binary from a python3.2 project I am getting the following runtime error:

/project/dist/project/distutils/__init__.py:13: UserWarning: The virtualenv distutils package at %s appears to be in the same location as the system distutils?
Traceback (most recent call last):
  File "/home/chrish/.virtualenvs/project/lib/python3.2/distutils/__init__.py", line 19, in <module>
    import dist
ImportError: No module named dist

Correspondingly there are several distutils entries in the missing modules section of the cxfreeze output:

? dist imported from distutils
? distutils.ccompiler imported from numpy.distutils.ccompiler
? distutils.cmd imported from setuptools.dist
? distutils.command.build_ext imported from distutils
? distutils.core imported from numpy.distutils.core
...

I've tried forcing distutils to be included as a module, by both importing it in my main python file and by adding it to a cxfreeze setup.py as:

options = {"build_exe": {"packages" : ["distutils"]} },

Neither approach worked. It seems likely that I've somehow broken the virtualenv [as distutils seems fundamental and the warning regarding the location of distutils], repeating with a clean virtualenv replicated the problem.

It may be worth noting that I installed cx-freeze by running $VIRTUAL_ENV/build/cx-freeze/setup.py install as it doesn't install cleanly in pip.

Answer 1

Summarising my comments:

The copy of distutils in the virtualenv is doing some bizarre things which confuse cx_Freeze. The simple workaround is to freeze outside a virtualenv, so that it uses the system copy of distutils.

On Ubuntu, Python 2 and 3 co-exist happily: just use python3 to do anything with Python 3. E.g. to install cx_Freeze under Python 3: python3 setup.py install.

Answer 2

Found another workaround which enables you to still use a virtualenv when freezing.

The workaround is to exclude distutils and add the package from the original interpreter (not from the virtualenv) manually.

# contents of setup.py
from cx_Freeze import setup, Executable

import distutils
import opcode
import os

# opcode is not a virtualenv module, so we can use it to find the stdlib; this is the same
# trick used by distutils itself it installs itself into the virtualenv
distutils_path = os.path.join(os.path.dirname(opcode.__file__), 'distutils')
build_exe_options = {'include_files': [(distutils_path, 'distutils')], "excludes": ["distutils"]}

setup(
    name="foo",
    version="0.1",
    description="My app",
    options={"build_exe": build_exe_options},
    executables=[Executable("foo_main.py", base=None)],
)

Credit to Bruno Oliveira for the answer on github
Full answer in gist: https://gist.github.com/nicoddemus/ca0acd93a20acbc42d1d

Answer 3

I have found a workaround about the distutils problem when freezing from within a virtualenv that might help others.

First make sure to exclude distutils from your build:

build_exe_options = {'excludes': ['distutils']}

Second declare this function in your setup.py file:

def copy_distutils_to_build_dir(build_dir):
    # the code below was obtained from the distutils.py file created by
    # virtualenv
    import opcode
    dirname = os.path.dirname
    distutils_path = os.path.join(os.path.dirname(opcode.__file__), 'distutils')
    target_dir = os.path.join(build_dir, 'distutils')
    if os.path.isdir(target_dir):
        shutil.rmtree(target_dir)
    shutil.copytree(distutils_path, target_dir)

Finally, call the function after calling setup() in your setup.py:

setup(...)
copy_distutils_to_build_dir(join('build', 'exe.win32-3.4'))

This will copy the entire distutils package from the original interpreter to the directory containing your frozen executable.

Hackish and ugly, but it works. I would love to hear ideas for improvement though.

Answer 4

One problem is that distutils/__init__.py in your venv tries to do an implicit relative import (import dist instead of the correct from distutils import dist), so that would be the first thing to fix. How was the venv created? Where does distutils/__init__.py come from?

Answer 5

It' s been a while now but I came across the same issue. I was able to solve it by copying the distutils package from the local Python library to the virtualenv library. I don' t know yet about side-effects. Everything seems to work well.