You have configuration management? Your hosts don’t need hostnames.

I often get asked how to automate host naming and/or DNS records using Chef. In fact, there was an individual in IRC today who asked some variation of the same question I always get:

Currently I set hostnames on my nodes by looking up the ip from ifconfig and doing a reverse DNS lookup on that IP. It turns out this is painful since my upstream never sets the rDNS correctly without nagging. I’m thinking of just building a “nodes” data bag with IP -> hostname mapping. Is there a better way to do this?

Often I’ve answered people by telling them that if they’re using configuration management, the names of their hosts are completely irrelevant. They shouldn’t even bother adding them to DNS or to even care what their names are. I thought I’d explain my rationale behind this and why host names are largely unnecessary if you’re using a configuration management tool like Chef. Continue reading →

Cooking with Opscode Chef on Windows: The Vagrant Edition

A month ago, I presented a webinar entitled “Cooking on Windows with Chef” that demonstrates the power of Opscode Chef on Windows. If you missed the webinar, you can watch that recording here.

One major way in which Windows has lagged Unix/Linux is in the desktop-based virtual machine development model using tools like VirtualBox and Vagrant. Vagrant, if you’re not already familiar with it, allows you to bring up and tear down development environments very quickly, and provision (configure) them using the same Chef cookbooks with which you’d configure your actual production environments. To that end, a bunch of folks have released an updated version of the vagrant-windows plugin, which adds WinRM and native shared folder support between Windows guests and the host operating system. Vagrant-windows has actually been around for a while, but had to be updated to deal with the API changes between Vagrant 1.1 and 1.2. This took a significant amount of work. Continue reading →

Why don’t more people go into tech? (Hint: It’s not an education problem.)

Ryan Holmes, the CEO of Vancouver social media startup HootSuite, wrote a column in today’s Financial Post entitled “Why Canada is failing at tech“. Holmes basically asserts that Canadians are “failing” at technology because the country isn’t graduating enough computer science and engineering talent to fill the available job openings. I don’t think Holmes has gone deep enough in his analysis. Why aren’t many people choosing computer science and engineering as career paths and the “jobs of tomorrow”? The answer to the question, I think, is pretty simple: it’s actually not a very nice job being a software developer. Continue reading →

Chef Cookbook Testing and Continuous Integration

I’ve been really busy working with our customers on becoming better Chefs, but I was recently invited by Kevin Karwaski of the Chef-Boston meetup group to give a presentation on the state of Chef cookbook testing today. Here are my slides: I hope the video will be up soon.

By the way, I renamed the demonstration cookbook from “sauceproxy” to “sauceconnect” in preparation for uploading it to the community site. Having unit and acceptance tests that I could run — both manually and via Travis — to verify that I’d done the search-and-replace properly really helped!

Technology Preview: Chef Cookbook integration testing with Test Kitchen 1.0

Chris Kimball of Cook's Illustrated at the American Museum of Natural History. (CC licensed.)

Chris Kimball of Cook’s Illustrated at the American Museum of Natural History. (CC-Att-NC)

How do you test your Chef cookbooks without firing up a real machine and uploading the recipes to a Chef Server?

I get asked this question all the time, especially after I’ve taught the basics of Chef at Opscode’s public training classes. For unit testing, there is ChefSpec — RSpec plus Chef primitives to allow you to make assertions about your recipes. However, that only goes so far without actually converging a real node & running external tests on the services that were configured.

Last year, Opscode released Test Kitchen, allowing you to use workstation-based virtualization (in the form of VirtualBox) to fire up test nodes, converge them, and run Minitests and Cucumber behavioural-driven development (BDD) tests on them after the converge. In this article, I’ll show you how to set up Test Kitchen 1.0 with Vagrant 1.1 to write and run integration tests.

Continue reading →

Reviving veewee after Vagrant 1.1

Recently, Mitchell Hashimoto released a major rewrite of Vagrant, the tool that lets you build, provision, and rebuild virtualized development environments at the click of a button. While the rewrite is great — it has many new features, chief amongst them the ability to use virtualization providers other than VirtualBox — it was a major architectural change, and this broke many tools that work with Vagrant. One of them is veewee, a popular tool from Patrick Debois that allows anyone to rebuild fresh Vagrant boxes. Here’s how I fixed my veewee installation for now. Continue reading →

Autoscaling Builds with Jenkins’ EC2 Plugin and Chef

One of my last projects at SecondMarket was to automate and rebuild the Jenkins infrastructure. We’d previously had a static setup in the NYC office with a build master and three slaves that ran all the time, but this handled developer check-in storms very poorly. For example, when developers were trying to make code cutoff for a feature, many builds would be queued for lack of available executors. But at other times, these agents would be completely idle. It made more sense to move the entire setup to the cloud and implement some kind of auto-scaling for Jenkins. Continue reading →