How Yipit Deploys Django - Yipit Django Blog

If you’re managing your own servers, and you don’t use a tool like Chef, you’re crazy. It’s just that simple.

We’ve been using Chef here at Yipit for about 6 months, and when I think about provisioning a new server with our old load book, I cringe.

There were some pretty high upfront costs to learning Chef, especially since no one here had any real Ruby experience (Chef is written in Ruby), but the time we invested into getting set up with Chef has been 100% worth it.

I’m hoping this post will help people get up and running with Django and Chef as quickly as possible. Opscode has their own django-quick-start repository, but I think it’s too complex if you’re not familiar with Chef. This tutorial will cover:

Getting set up with Opscode, a hosted chef-server
Installing Ruby, Chef, and some knife plugins
Setting up your chef-repo and installing a Django/github quickstart cookbook
Using a Python script to deploy to ec2

If you don’t understand what any of those things mean, Opscode maintains a pretty good wiki.

Before We Start

This tutorial makes the following assumptions about your systems:

Locally, you run a Mac.
Remotely, you want to use Ubuntu on AWS
Your AWS account has security groups named “web” and “linux” that allow access on ports 22 and 80
The private key for the AWS ssh key you define in add_server.py (we’ll get to that later) is located in ~/.ssh/ on your local machine
Your Django app contains a pip requirements file at conf/external_apps.txt (and that file contains gunicorn)
Your Django app contains a gunicorn configuration file at conf/gunicorn/gunicorn.conf
You’re interested in learning about Chef and managing your own servers. Seriously. If you’re just looking for a simple deployment solution, use Heroku. If you want to manage and automate your own infrastructure, use Chef

If you’re not familiar with AWS security groups and key pairs, this tutorial should help. I’ve also provided a simple “Hello World” Django application that I use as an example.

Get the Gems

I like using homebrew for everything, so even though your mac comes with ruby…

brew install ruby
gem install -n /usr/local/bin/ chef
gem install knife-ec2
gem install knife-github-cookbooks

Set up a chef-repo

We’re going to be adding some python scripts to help us manage our servers, so I like to set up my chef-repo in a virtualenv. From wherever you keep your virtualenvs…

virtualenv chef-env
cd chef-env
git clone https://github.com/opscode/chef-repo.git
cd chef-repo

Now you have the barebones Chef repository distributed by opscode.

Get started with opscode and your knife.rb file

Next you’ll want to get setup with Opscode to manage your chef server. You can create an account at opscode.com.

After you’ve signed up, download your private key. You can download it by clicking on your username on the top right of the console, and then following the link to “get private key”.

Next you’ll need to set up an organization and download the organization validator key. You’ll also want to to download the knife.rb file that opscode will generate for you.

Lastly, throw it all in a .chef directory in your chef-reop.

You should also append this code to your knife.rb:

This changes the amount of time knife will wait for ec2 to spawn a new server for you. I’ve found that sometimes the default wait time isn’t long enough for ebs-backed instances.

Setting up your project

Next we need to set up chef to actually do something. Let’s start by installing a Django quick start cookbook I put together. This cookbook will install a django application behind nginx and gunicorn, which will be managed by supervisor. You should read through the commented code for an explanation of how it works.

knife cookbook github install Yipit/djangoquickstart-cookbook

This cookbook has a dependency on the python cookbook, so we’ll want to install that too.

knife cookbook site install python

Note that “site” version of the install command will handle dependencies automatically, while the github command will not. When you use “site”, however, you’re limited to cookbooks distributed by opscode.

The next thing we’ll want to do is define a role. In your chef-repo, put something like this in roles/web.rb:

Get Some Python In there

Knife has a pretty good plugin system, but it requires you to write Ruby. Writing Ruby is worth it in the cookbooks because it’s the only option, but for helper scripts, I just use python. Aside from being more comfortable with the language, familiarity with the libraries is a huge plus. Here’s a script similar to the one we use to add servers at Yipit:

Fill in your AWS keys and put this code into a file called add_server.py. This script requires boto and pychef, so from within your virtualenv:

pip install boto pychef

I keep this file in right in the chef repo.

The last thing we need to do before we get started is upload our roles and cookbooks to the chef server.

knife cookbook upload --all
knife role from file roles/web.rb

Now, if you run the add_server.py script, you should be able to bootstrap your own web server running your django application through gunicorn behind nginx. The add server script does restart the machine, so you may need to wait a minute or two.

Hopefully you can visit your new web server and see your application. If you have any issues, be sure to leave a comment and I’ll try my best to help you out.

Zach Smith is Technical Product Manager at Yipit.

To find out about future posts, you can follow along using