Chris Kimball of Cook’s Illustrated at the American Museum of Natural History. (CC-Att-NC)
How do you test your Chef cookbooks without firing up a real machine and uploading the recipes to a Chef Server?
I get asked this question all the time, especially after I’ve taught the basics of Chef at Opscode’s public training classes. For unit testing, there is ChefSpec — RSpec plus Chef primitives to allow you to make assertions about your recipes. However, that only goes so far without actually converging a real node & running external tests on the services that were configured.
Last year, Opscode released Test Kitchen, allowing you to use workstation-based virtualization (in the form of VirtualBox) to fire up test nodes, converge them, and run Minitests and Cucumber behavioural-driven development (BDD) tests on them after the converge. In this article, I’ll show you how to set up Test Kitchen 1.0 with Vagrant 1.1 to write and run integration tests.
Before we begin, I have to warn you that Test Kitchen 1.0.0 is a major rewrite and still under heavy development — it is currently in alpha, so there may will still be bugs. In particular, the directions in the README on the Test Kitchen GitHub refer to the use of Bundler, and this is no longer required or recommended.
One of the major new features in Test Kitchen is to abstract out the drivers for different virtualization providers from the actual Test Kitchen framework. This allows you to run your Kitchen tests on any virtualization provider: you could use Kitchen to converge on a VirtualBox machine when you’re on an airplane, but when you’re in the office, you could converge on an Amazon EC2 instance. For this demonstration, though, we’re going to continue to use VirtualBox for simplicity.
I’m going to assume you have both Chef and VirtualBox installed on your workstation. If you’ve used Opscode’s recommended Omnibus installer, Chef will be installed in /opt/chef (or C:\chef if on Windows) so those are the paths I’m going to refer to.
Vagrant Installation
Vagrant is a framework for virtualization providers that allows you to rebuild environments from scratch, using a base image and applying the same provisioning logic you would normally run in your production environment to manage your infrastructure. For Puppet, that might mean your Puppet manifests; for Chef, it means your Chef cookbooks. There’s more information about Vagrant at vagrantup.com, but suffice it to say that we’ll be using it as the abstraction layer to VirtualBox, so go ahead and download & install that.
Berkshelf Installation, Part 1
Berkshelf is a cookbook artifact manager, which obviates you from having to store “upstream” or community cookbooks on your local machine, or even in your own source control management system. Test Kitchen uses it to satisfy the dependencies for the cookbook under test, so we’re going to install that as well.
There are two parts to the installation. The first part is to install the Berkshelf plugin for Vagrant, so that it knows to call out to Berkshelf when running the provisioning phase. So go ahead and do that:
$ vagrant plugin install berkshelf-vagrant --plugin-version 1.0.6 Installing the 'berkshelf-vagrant' plugin. This can take a few minutes... Installed the plugin 'berkshelf-vagrant (1.0.6)'!
Note: berkshelf-vagrant 1.1.0 won’t work until this patch gets merged to the Kitchen Vagrant driver. (Remember what I said about alpha software?)
We’ll install the other part of Berkshelf after we’ve installed Test Kitchen.
Test Kitchen Installation
Let’s install the pre-release version of Test Kitchen into Chef’s embedded Ruby:
$ sudo /opt/chef/embedded/bin/gem install test-kitchen --pre Successfully installed test-kitchen-1.0.0.alpha.4 1 gem installed Installing ri documentation for test-kitchen-1.0.0.alpha.4... Installing RDoc documentation for test-kitchen-1.0.0.alpha.4...
We also need a virtualization driver for Test Kitchen; by default, no drivers are installed. Let’s install the Vagrant driver, since that’s what we’re going to use:
$ sudo /opt/chef/embedded/bin/gem install kitchen-vagrant Succesfully installed kitchen-vagrant-0.7.4 1 gem installed Installing ri documentation for kitchen-vagrant-0.7.4... Installing RDoc documentation for kitchen-vagrant-0.7.4...
Berkshelf Installation, Part 2
We also need to make Test Kitchen aware that we’re using Berkshelf for cookbook artifact management, so let’s do that:
$ sudo /opt/chef/embedded/bin/gem install berkshelf Successfully installed berkshelf-1.3.1 1 gem installed Installing ri documentation for berkshelf-1.3.1... Installing RDoc documentation for berkshelf-1.3.1...
That’s it! Now we’re ready to roll.
Running Test Kitchen Tests
Let’s not go into how to write tests for Test Kitchen just yet: let’s just get cooking using some pre-written Chef minitests in the Opscode bluepill cookbook. Let’s clone that from GitHub and do some testing.
$ git clone https://github.com/opscode-cookbooks/bluepill.git
The .kitchen.yml file describes all the platforms we’re interested in testing, how to set them up to serve as a sensible test fixture for our tests, and finally, what the test suite consists of. In this situation, the test suite just sets up a couple of required attributes, and adds a bluepill_test cookbook to the test harness. You’ll see the code for that cookbook under test/cookbooks/bluepill_test. The suite also adds the minitest-handler cookbook to the run list, which is how the minitests (located in that test cookbook under files/default/tests/minitest) will actually get run.
Ok, so let’s start up our kitchen:
$ /opt/chef/embedded/bin/kitchen test
-----> Starting Kitchen
-----> Cleaning up any prior instances of
-----> Destroying
Finished destroying (0m0.00s).
-----> Testing
-----> Creating
[kitchen::driver::vagrant command] BEGIN (vagrant up --no-provision)
Bringing machine 'default' up with 'virtualbox' provider...
[default] Importing base box 'canonical-ubuntu-12.04'...
I’ve elided the rest of the output as it’s very long, but basically what Kitchen will do is to start up and try to converge a VM for every platform you’ve defined in the .kitchen.yml, and throw an exception if it fails on any one. Neat, huh?
How Do I Write My Test Cases?
The Chef Minitest framework provides you a lot of assertions out of the box; look at the documentation on GitHub for some examples of how to write regular Minitest-style testcases, or spec-style testcases. There are also custom assertions bundled with Minitest.
As with many open-source projects, the README can be a bit lacking with respect to the features currently in Minitest Chef. The default_test.rb in the Minitest Chef Handler project shows a very comprehensive list of tests that can be made.
Wrap-Up
I’ve shown you how to integration test your cookbooks using Opscode Test Kitchen 1.0 Alpha, Berkshelf and Vagrant with VirtualBox as the underlying virtualization technology. Obviously, this is a field under heavy development, and some features could change between now and the actual release of Test Kitchen 1.0. But hopefully it’s enough to get you started.
Additional Resources
Joshua Timberman has a great two-part series on his blog, delving into more detail about Test Kitchen 1.0.