Elegantly handle site-specific settings/configuration in svn/hg/git/etc?

Question

I\'ve been looking for a better way to deal with site-specific settings (in this case, the django settings.py file).

The settings.py structure and fields are fairly

Answer 1

Lets separate those two distinct problems: 1) managing site-specific settings and 2) managing secrets.

1) Site-specific settings

Version everything (except secrets), even developer-specific settings.

With Django and a lot of other softwares, the configuration file is a piece of executable code, which makes it easy to load common configuration settings and override whatever needs to be overridden. This way you can stay DRY.

# settings_prod.py
from settings_base import *
... # override whatever needs to be overridden for production environment

So now you have settings_base.py, settings_prod.py, settings_dev.py, settings_developper_john.py, etc. How do you tell Django which one to use?

Deploying the appropriate settings file to the server is a task for the deployment script, I believe. The deployment script would know that you're deploying to host prod17 which is a production server, so it would generate on the fly a settings.py file that would look like this:

# settings.py (generated by deployment script)
from settings_prod import *

Another solution is to have that logic in a generic settings.py: it could read an environment variable or get the host name (or apply any other logic) and load the appropriate settings module:

# settings.py
import os
if os.environ["MY_APP_ENV"] == "prod":
    from settings_prod import *
elif ...

My favorite solution for Django settings is described here.

For any other software that is not as flexible with it's configuration file, the best option is probably to have the deployment script generate the configuration file, possibly using templates (tools like Chef or Puppet make this easy). This allows you to stay DRY: for example, say a software requires a flat config.ini file, then the deployment script could read a common.ini and a production.ini file, mix them together appropriately and produce a config.ini ready to be deployed to production.

Managing secrets

First of all, do not store your passwords in a version control system. :-)

One solution for managing secrets is to have the deployment script transfert the secrets. For example, bob is responsible for the deployment of web applications, he knows the password to the database, so when he launches the deployment script, he is prompted for the database password, and the script transfers it to the server. Or the deployment script simply reads the password in a file on bob's computer and transfers it. This is probably the most common solution. It's fine in most cases.

              secrets
deployer ================> server

If you need to automate the creation of VMs and you do not want the automated-deployer to know any secret, then you could include the secrets in the VM-image. Of course someone must include the secrets in the VM image in the first place.

                  VM image including secrets
human deployer -------------------------------+
                                              |
                                              |
                    image_name                v
automated deployer ==============> Cloud Service ========> VM including secrets

The problem with this solution is that you need to generate a new VM image every time any secret changes. If you want to avoid that, then you might want a "secret-server": a server to manage every other server's secrets. Then the only secret you need to include in the VM image is the bootstrap secret needed to connect to the "secret-server".

step 1:

               VM image including bootstrap secret
human deployer -----------------------------------+
                                                  |
                                                  |
                    image_name                    v
automated deployer ==================> Cloud Service ========> VM including secrets


step 2:

    bootstrap secret
   ==================>
VM                     Secret Server
   <==================
         secrets

For example, the secret server could be a Chef server, the secrets could be store in encrypted data bags, and the bootstrap secret would be the key to decrypt these bags.