Semantic Chef: Writing Readable Recipes

The second in a short series of Chef related blog posts

Why Chef is awesome

Programmable in Ruby
Repeatable recipes define your infrastructure

Why Chef is painful

Programmable in Ruby
Slow ‘REPL’ to test changes

Ruby lets you do just about anything. It’s a programming language, that’s what it is supposed to do. Chef’s DSL is concise, readable and extensible. The power of Chef comes by allowing you to program in Ruby. When mixing Chef’s DSL with the power of ruby, things can get out of hand if you are not careful.

Cookbook Examples

Simple

The Opscode examples show nice things. As a systems person it is very easy to understand what is going on.

package "ntp" do
    action [:install]
end

template "/etc/ntp.conf" do
    source "ntp.conf.erb"
    variables( :ntp_server => "time.nist.gov" )
    notifies :restart, "service[ntpd]"
end

service "ntpd" do
    action [:enable,:start]
end

Messy

Add some more complex logic in and we get a bit messier.

if platform_family?("rhel") and node['python']['install_method'] == 'package'
  pip_binary = "/usr/bin/pip"
elsif platform_family?("smartos")
  pip_binary = "/opt/local/bin/pip"
else
  pip_binary = "/usr/local/bin/pip"
end

# Ubuntu's python-setuptools, python-pip and python-virtualenv packages
# are broken...this feels like Rubygems!
# http://stackoverflow.com/questions/4324558/whats-the-proper-way-to-install-pip-virtualenv-and-distribute-for-python
# https://bitbucket.org/ianb/pip/issue/104/pip-uninstall-on-ubuntu-linux
remote_file "#{Chef::Config[:file_cache_path]}/distribute_setup.py" do
  source node['python']['distribute_script_url']
  mode "0644"
  not_if { ::File.exists?(pip_binary) }
end

execute "install-pip" do
  cwd Chef::Config[:file_cache_path]
  command <<-EOF
  #{node['python']['binary']} distribute_setup.py --download-base=#{node['python']['distribute_option']['download_base']}
  #{::File.dirname(pip_binary)}/easy_install pip
  EOF
  not_if { ::File.exists?(pip_binary) }
end

Inscrutable

Sure you can figure out what it does… eventually.

if node["platform"] == "windows"
    existence_check = :exists?
# Where will also return files that have extensions matching PATHEXT (e.g.
# *.bat). We don't want the batch file wrapper, but the actual script.
    which = 'set PATHEXT=.exe & where'
    Chef::Log.debug "Using exists? and 'where', since we're on Windows"
else
    existence_check = :executable?
    which = 'which'
    Chef::Log.debug "Using executable? and 'which' since we're on Linux"
end

# COOK-635 account for alternate gem paths
# try to use the bin provided by the node attribute
if ::File.send(existence_check, node["chef_client"]["bin"])
  client_bin = node["chef_client"]["bin"]
  Chef::Log.debug "Using chef-client bin from node attributes: #{client_bin}"
# search for the bin in some sane paths
elsif Chef::Client.const_defined?('SANE_PATHS') && (chef_in_sane_path=Chef::Client::SANE_PATHS.map{|p| p="#{p}/chef-client";p if ::File.send(existence_check, p)}.compact.first) && chef_in_sane_path
  client_bin = chef_in_sane_path
  Chef::Log.debug "Using chef-client bin from sane path: #{client_bin}"
# last ditch search for a bin in PATH
elsif (chef_in_path=%x{#{which} chef-client}.chomp) && ::File.send(existence_check, chef_in_path)
  client_bin = chef_in_path
  Chef::Log.debug "Using chef-client bin from system path: #{client_bin}"
else
  raise "Could not locate the chef-client bin in any known path. Please set the proper path by overriding node['chef_client']['bin'] in a role."
end

node.set["chef_client"]["bin"] = client_bin

# libraries/helpers.rb method to DRY directory creation resources
create_directories

case node["chef_client"]["init_style"]
when "init"

  #argh?
  dist_dir, conf_dir = value_for_platform_family(
    ["debian"] => ["debian", "default"],
    ["fedora"] => ["redhat", "sysconfig"],
    ["rhel"] => ["redhat", "sysconfig"],
    ["suse"] => ["suse", "sysconfig"]
  )
# .. snip 200 lines of case statement

What’s Wrong

I dont like this. When I look at a recipe I want to be able to easily understand what it is doing. It is a recipe, not an annotated thesis. It should be a simple browsable description of your intentions. Real cookbooks don’t describe the minutia of each step, they tell you the general steps and leave you to figure out the details.

How?

Get your logic out of your recipe, put it in a library.
Abstract connected DSL blocks to LWRPs, control will be simpler
Use name attributes carefully.
Give friendly names to referenced node attributes.

Why?

Readability
Testability
Development Speed
Readability

Developers who aren’t that familiar with Ruby or Chef can still easily read recipe code. It is designed to be readable and declarative. The recipes should define the pieces of your infrastructure, not the edge cases.

Testability

It is hard to test Chef directly. Usually it just results in you running all of the code and testing the system’s final state. When you separate your logic from your Chef code, you separate your concerns. You now have logic to which you can easily apply standard Ruby testing methods. The rest of the recipe is now just Chef code that only needs to be checked for regressions.

Development Speed

Now that you have your logic separate, you can write it just like regular Ruby. You don’t need to waste all of your time waiting for VMs to spin up and compile packages. The only time you need to run Chef is to verify your DSL blocks are set up correctly.

Downsides

You have to learn about library files
You have to write tests
You have to name more things.
You’ll have to learn about when Chef loads and runs libraries
You’ll have to learn more Ruby
Less elegant code to force calling from Library files.

Next Step

Chef manipulates Infrastructure as Code, its about time you started treating your infrastructure code like your application code.

Note : Discussions of this could easily devolve into the MVC “where do I put my logic” debate.

Andrew Gross is a Developer at Yipit, you can find him as @awgross on Twitter

Yipit Django Blog