How are people testing opsworks cookbooks?

Question

I have a fairly complex deployment of dynamically configured instances of a rail apps on nginx using passenger that I currently use Capistrano for. I am trying to automate the e

Answer 1

There are a lot of useful suggestions on this page but I would HIGHLY recommend everyone check out Mike Greiling's blog post Simplify OpsWorks Development With Packer and his github repo opsworks-vm which help you to mock the entire opsworks stack including the install of the opsworks agent so you can also test app deploy recipes, multiple layers, multiple instances at the same time, etc . It is extremely impressive.

Quick Start on Ubuntu 14.04

NOTE: This can NOT be done from an ubuntu virtual machine because virtualbox does not support nested virtualization of 64-bit machines.

1. Install ChefDK
   1. mkdir /tmp/packages && cd /tmp/packages
   2. wget https://opscode-omnibus-packages.s3.amazonaws.com/ubuntu/12.04/x86_64/chefdk_0.8.1-1_amd64.deb
   3. sudo dpkg -i chefdk_0.8.0-1_amd64.deb
   4. cd /opt/chefdk/
   5. chef verify
   6. which ruby
   7. echo 'eval "$(chef shell-init bash)"' >> ~/.bash_profile && source ~/.bash_profile
2. Install VirtualBox
   1. echo 'deb http://download.virtualbox.org/virtualbox/debian vivid contrib' > /etc/apt/sources.list.d/virtualbox.list
   2. wget -q https://www.virtualbox.org/download/oracle_vbox.asc -O- | sudo apt-key add -
   3. sudo apt-get update -qqy
   4. sudo apt-get install virtualbox-5.0 dkms
3. Install Vagrant
   1. cd /tmp/packages
   2. wget https://dl.bintray.com/mitchellh/vagrant/vagrant_1.7.4_x86_64.deb
   3. sudo dpkg -i vagrant_1.7.4_x86_64.deb
   4. vagrant plugin install vagrant-berkshelf
   5. vagrant plugin install vagrant-omnibus
   6. vagrant plugin list
4. Install Packer
   1. mkdir /opt/packer && cd /opt/packer
   2. wget https://dl.bintray.com/mitchellh/packer/packer_0.8.6_linux_amd64.zip
   3. unzip packer_0.8.6_linux_amd64.zip
   4. echo 'PATH=$PATH:/opt/packer' >> ~/.bash_profile && source ~/.bash_profile
5. Build Mike Greiling's opsworks-vm virtualbox image using Packer
   1. mkdir ~/packer && cd ~/packer
   2. git clone https://github.com/pixelcog/opsworks-vm.git
   3. cd opsworks-vm
   4. rake build install
   5. This will install a new virtualbox vm to ~/.vagrant.d/boxes/ubuntu1404-opsworks/

To mock a single opsworks instance, create a new Vagrantfile like so:

Vagrant.configure("2") do |config|
  config.vm.box = "ubuntu1404-opsworks"
  config.vm.provision :opsworks, type: 'shell', args: 'path/to/dna.json'
end

The dna.json file path is set relative to the Vagrantfile and should contain any JSON data you wish to send to OpsWorks Chef.

For example:

{
  "deploy": {
    "my-app": {
      "application_type": "php",
      "scm": {
        "scm_type": "git",
        "repository": "path/to/my-app"
      }
    }
  },
  "opsworks_custom_cookbooks": {
    "enabled": true,
    "scm": {
      "repository": "path/to/my-cookbooks"
    },
    "recipes": [
      "recipe[opsworks_initial_setup]",
      "recipe[dependencies]",
      "recipe[mod_php5_apache2]",
      "recipe[deploy::default]",
      "recipe[deploy::php]",
      "recipe[my_custom_cookbook::configure]"
    ]
  }
}

To mock multiple opsworks instances and include layers see his AWS OpsWorks "Getting Started" Example which includes the stack.json below.

Vagrantfile (for multiple instances)

Vagrant.configure("2") do |config|

  config.vm.box = "ubuntu1404-opsworks"

  # Create the php-app layer
  config.vm.define "app" do |layer|

    layer.vm.provision "opsworks", type:"shell", args:[
      'ops/dna/stack.json',
      'ops/dna/php-app.json'
    ]

    # Forward port 80 so we can see our work
    layer.vm.network "forwarded_port", guest: 80, host: 8080
    layer.vm.network "private_network", ip: "10.10.10.10"
  end

  # Create the db-master layer
  config.vm.define "db" do |layer|

    layer.vm.provision "opsworks", type:"shell", args:[
      'ops/dna/stack.json',
      'ops/dna/db-master.json'
    ]

    layer.vm.network "private_network", ip: "10.10.10.20"
  end
end

stack.json

{
  "opsworks": {
    "layers": {
      "php-app": {
        "instances": {
          "php-app1": {"private-ip": "10.10.10.10"}
        }
      },
      "db-master": {
        "instances": {
          "db-master1": {"private-ip": "10.10.10.20"}
        }
      }
    }
  },
  "deploy": {
    "simple-php": {
      "application_type": "php",
      "document_root": "web",
      "scm": {
        "scm_type": "git",
        "repository": "dev/simple-php"
      },
      "memcached": {},
      "database": {
        "host": "10.10.10.20",
        "database": "simple-php",
        "username": "root",
        "password": "correcthorsebatterystaple",
        "reconnect": true
      }
    }
  },
  "mysql": {
    "server_root_password": "correcthorsebatterystaple",
    "tunable": {"innodb_buffer_pool_size": "256M"}
  },
  "opsworks_custom_cookbooks": {
    "enabled": true,
    "scm": {
      "repository": "ops/cookbooks"
    }
  }
}

For those not familiar with vagrant you just do a vagrant up to start the instance(s). Then you can modify your cookbook locally and any changes can be applied by re-running chef against the existing instance(s) with vagrant provision. You can do a vagrant destroy and vagrant up to start from scratch.