PyPI is slow. How do I run my own server?

Question

When a new developer joins the team, or Jenkins runs a complete build, I need to create a fresh virtualenv. I often find that setting up a virtualenv with Pip and a large nu

Answer 1

I recently installed devpi into my development team's Vagrant configuration such that its package cache lives on the host's file system. This allows each VM to have its own devpi-server daemon that it uses as the index-url for virtualenv/pip. When the VMs are destroyed and reprovisioned, the packages don't have to be downloaded over and over. Each developer downloads them one time to build their local cache for as long as they live on the host's file system.

We also have an internal PyPi index for our private packages that's currently just a directory being served by Apache. Ultimately, I'm going to convert that to a devpi proxy server as well so our build server will also maintain a package cache for our Python dependencies in addition to hosting our private libraries. This will create an additional buffer between our development environment, production deployments and the public PyPi.

This seems to be the most robust solution I've found to these requirements to date.