pip: Why sometimes installed as egg, sometimes installed as files

Question

Where can you force pip to install as "flat" and not as "egg".

For me it seems random. Sometimes it gets installed as egg, sometime as flat.

pip help install shows only an option --egg which forces an egg installation. But I could not find a --flat option.

The packages are from an own pypiserver, and uploaded like this:

python setup.py sdist upload -r internal

Output of pip during install:

Best match: foo-client 2015.2
Downloading https://installserver:40443/pypi/packages/foo_client-2015.2.tar.gz
Processing foo_client-2015.2.tar.gz
Writing /home/bar_eins_daad/tmp/easy_install-z20B7b/foo_client-2015.2/setup.cfg
Running foo_client-2015.2/setup.py -q bdist_egg --dist-dir /home/bar_eins_daad/tmp/easy_install-z20B7b/foo_client-2015.2/egg-dist-tmp-GO1snX

I don't know why bdist_egg gets used here. Does it force creating an egg install?

The setup.py does use setuptools not distutils.

The package on our pypiserver looks like this:

tar -tzf packages/foo_client-2015.3.tar.gz

content:

foo_client-2015.2/
foo_client-2015.2/foo_client.egg-info/
foo_client-2015.2/foo_client.egg-info/SOURCES.txt
foo_client-2015.2/foo_client.egg-info/top_level.txt
foo_client-2015.2/foo_client.egg-info/dependency_links.txt
foo_client-2015.2/foo_client.egg-info/PKG-INFO
foo_client-2015.2/setup.cfg
foo_client-2015.2/PKG-INFO
foo_client-2015.2/foo_client/
foo_client-2015.2/foo_client/models.py
...

Background

pip seems to sometimes install packages several times if zipped eggs are installed.

Update

I found under which condition the package gets installed as egg: if it gets installed via python setup.py develop (it is an install_requires dependency).

If I use pip install foo_client it gets installed flat (the way I want it).

Update2

The very ugly part: If the egg gets installed, the old version installed flat does not get removed.

Version: pip 1.5.6

Answer 1

This does not solve the question why I get sometimes zipped eggs, and sometimes not. But it helps.

You can use this in your ~/.distutils.cfg to avoid installation of zipped eggs:

[easy_install]
zip_ok = False

Answer 2

If you are the author of the package, you can use the flag zip_safe=False in setup.py.

setup(
    name = "HelloWorld",
    ...
    zip_safe = False,
)

If you are a user who wants to improve the package, you can install it by pip install -e foo_package. The option -e or --editable installs a project in editable mode (i.e. setuptools "develop mode"), not zipped. It creates a link from source into site-packages and compiles .../bin scripts, but doesn't copy the source there site-packages. Those packages can not be updated automatically and it is the main reason why it is not intended as usual way of installing packages, but only for those that need to be customized or fixed.

Edit: Django is a typical framework that requires zip_safe=False for its applications, because they are not a pure Python, but they contain also templates with html, css, i18n resources etc. Is your question related to Django?

Answer 3

I was having this egg-only install problem, and it turned out I had failed to git add the __init__.py in the root of my package. It was driving me crazy that this would work:

pip install .

...but this would fail:

mkdir /tmp/piptest
cd /tmp/piptest
git clone $OLDPWD .
pip install .

It was hard to notice the difference using diff -r . $OLDPWD because there are so many non-committed pyc files and development tool scripts.

This is probably not the answer for this OP, but I hope it helps someone who Googles "pip only installing the egg" as I did.

Answer 4

I had the same issue as @guettli and solved it by unzipping and extracting the archive first then running:

pip -e install /srv/mypkg-1.1.0

where /srv/mypkg-1.1.0/ is the top level directory of project/package that has a setup.py file in it.

mypkg-1.1.0 was installed in site-packages and mypkg.py was listed in [virtualenv]/bin

note: '-e' flag is optional.

Thanks

Answer 5

Some docs say:

For maximum performance, Python packages are best installed as zip files.

and

You can pass a True or False value for the zip_safe argument to the setup() function, or you can omit it. If you omit it, the bdist_egg command will analyze your project’s contents to see if it can detect any conditions that would prevent it from working in a zipfile.

So it's probably fine unless... it's not fine in your case. My project tried to read files and failed due to the zipping.

Notably, this only happened with python setup.py install, not pip install . @guettli's fix worked fine, but I put it in setup.cfg:

[easy_install]
zip_ok = False