A simple Python deployment problem - a whole world of pain

Question

We have several Python 2.6 applications running on Linux. Some of them are Pylons web applications, others are simply long-running processes that we run from the command lin

Answer 1

I've been working on implementing this for our work projects. It's a few different parts involved.

First, we customize virtualenv.py using their bootstrap abilities to add in your own custom post-creation functions and flags. These allow us to define common types of projects and also gives us a single command to create a new virtualenv, checkout a project from the git repository, and install any requirements into the virtualenv using pip and requirements.txt files.

so our commands look like: python venv.py --no-site-packages -g $git_proj -t $tag_num $venv_dir

http://pypi.python.org/pypi/virtualenv http://pip.openplans.org/

Now that gets us through the initial checking out of an existing project. As we work and update the project we use fabric commands within each project to build releases and then to deploy them:

http://docs.fabfile.org/0.9.0/

I've got a fab command: make_tag which checks for unused commits, opens files that need version strings updated, builds and uploads sphinx docs, and then commits the final tag to the repository.

The flip side is a fab deploy command which will, over ssh, do a git co of the tag specified, run a pip update on any new requirements, run any database migrations needed, and then resets the web server if this is a web application.

Here's an example of the tagging function: http://www.google.com/codesearch/p?hl=en#9tLIXCbI4vU/fabfile.py&q=fabfile.py%20git%20tag_new_version&sa=N&cd=1&ct=rc&l=143

There are a ton of good fabric files you can browse through using the google code search. I know I cheat-sheeted a few for my own use.

It's definitely complicated and has several parts in order to get things running smooth. Once you get it running though, the flexibility and speed for things is just awesome.