Julian C. Dunn

Chris Kimball of Cook’s Illustrated at the American Museum of Natural History. (CC-Att-NC)

How do you test your Chef cookbooks without firing up a real machine and uploading the recipes to a Chef Server?

I get asked this question all the time, especially after I’ve taught the basics of Chef at Opscode’s public training classes. For unit testing, there is ChefSpec — RSpec plus Chef primitives to allow you to make assertions about your recipes. However, that only goes so far without actually converging a real node & running external tests on the services that were configured.

Last year, Opscode released Test Kitchen, allowing you to use workstation-based virtualization (in the form of VirtualBox) to fire up test nodes, converge them, and run Minitests and Cucumber behavioural-driven development (BDD) tests on them after the converge. In this article, I’ll show you how to set up Test Kitchen 1.0 with Vagrant 1.1 to write and run integration tests.

Continue reading…

Recently, Mitchell Hashimoto released a major rewrite of Vagrant, the tool that lets you build, provision, and rebuild virtualized development environments at the click of a button. While the rewrite is great — it has many new features, chief amongst them the ability to use virtualization providers other than VirtualBox — it was a major architectural change, and this broke many tools that work with Vagrant. One of them is veewee, a popular tool from Patrick Debois that allows anyone to rebuild fresh Vagrant boxes. Here’s how I fixed my veewee installation for now. Continue reading…

Tomorrow I’ll be joining Opscode as a senior consultant for Chef. My job responsibilities will be diverse, encompassing training, evangelism, and also working on projects for customers large and small.

I’m extremely excited to be working for a company whose product has been revolutionizing the job responsibilities of the traditional system administrator, and even those of the software engineer. It’s easier to break down the walls between operations and development when all your infrastructure is code, and Chef makes that a no-brainer. Frankly, it’s also more fun for everyone — yes, it’s possible for web operations to be fun again, just like it was back in 1996 when I got into this sort of thing.

I’m looking forward to working with all of the really smart people at Opscode, and, if you’re part of the Chef community, with you as well. See you around, maybe at a conference, training session, or just in IRC!

I was recently in San Diego giving a talk at LISA ’12 entitled “What Your CDN Won’t Tell You: Optimizing a News Website for Speed and Stability“. The paper was based on work my colleague Blake Crosby and I did at the Canadian Broadcasting Corporation to rearchitect their website & integrate it successfully with Akamai‘s EdgeSuite content delivery network, enabling it to serve over a million unique visitors daily with high availability.

I’m pleased to announce that the slides from the presentation are now available, and the video (eek!) will be shortly.

Hope everyone’s having a great holiday and I promise there will be more blog posts in the New Year.

Earlier this week I gave a talk at the Chef-NYC meetup about Shef, the Chef Shell. Although it was more of an interactive demo rather than a traditional presentation, I’ve put the slides up nonetheless.

I’ve also been meaning to post some notes from the 2012 Surge Conference that I attended at the end of September. Haven’t had much of a chance to write those up into something coherent, but I will soon. Meanwhile, check out the videos, which have just been posted.

Finally, I will be at DevOpsDays in a couple weeks — though not speaking. If you’re going, I’m looking forward to meeting you!

I’ve written a new post for the SecondMarket Engineering blog about our experiences setting up our own DNS servers in Amazon EC2. You can read it at engineering.secondmarket.com.

At SecondMarket, we’re moving towards a development model where not only are the deployments continuous but where deploys incur no downtime. Users should not notice if we take a portion of our servers out for maintenance, even if they’re logged into the site and have an active session. We decided to tackle this problem by persisting Java sessions to external storage. This allows another Jetty to take over serving of existing sessions if we decide to take down a Jetty for maintenance. Continue reading…

I‘ve been very busy at my job in system operations over at SecondMarket, trying to get our infrastructure in shape for many changes coming down the pipe. On the business side, the JOBS Act passed by Congress back in April means that the ban on general solicitation of accredited investors is being lifted, and so we expect to be able to grow our client base as a result.

More clients means more features needed to cater to them. On the technology side we have been working hard to deliver small packages of features faster, rather than in one large biweekly release: in other words, continuous delivery. I’m looking forward to the day when we can finally hand over the keys to engineering & have developers deploy whenever they want, using our Jenkins continuous integration system. Operations people have no business being a roadblock to software developers who want to get features out the door as quickly as possible. As long as the code is of high quality and doesn’t crash the servers, I’m comfortable with whatever gets deployed into production. It also means that engineers are 100% responsible for both the success and failure of their code — a simultaneous carrot & stick towards increasing quality.

The whole discussion around push-button deploys has led us purchasing an actual USB pushbutton. Made by a company called Dream Cheeky, this button — admittedly a little more flimsy than it appears in the picture — ships with only a Windows driver. Fortunately, someone has written a RubyGem and a Mac driver to interface with it. We’re taking the next logical step and making it possible to deploy with literally a button push. Continue reading…

I haven’t written much HTML since 1996. Back in those wild west days before CSS existed, we used <font> tags and <table>-based layouts to control how websites were presented to end-users. Although these ugly hacks limited the sophistication of web sites and later proved to be a barrier to the development of rich Internet applications, they were at least part of a standard: HTML 2.0.

Today, however, I think the state of markup on the Internet is far worse, despite the existence of this beast we call HTML5. To my utter shock, I discovered that HTML5 isn’t even what one could call a standard. On the contrary, HTML5 represents standards committees (and there are two — I’ll get into this later) throwing in the towel because of internecine fighting, to the enormous detriment of web, application and browser developers everywhere. The most poignant illustration of the problem is that there is no DTD for HTML5: you merely write <!DOCTYPE html> and you’re on your way. Nothing says “anything goes” better than “don’t even bother mechanically validating this because who the hell knows what’s valid?” Continue reading…

IPv6, the next version of the Internet’s addressing scheme, is back — and this time it’s here to stay.

Following up on last year’s World IPv6 Day, the Internet Society has organized World IPv6 Launch Day for June 6th. On this day, many major ISPs and corporations will permanently launch their IPv6 presence, in recognition of the fact that the world has now exhausted the IPv4 address space and must urgently migrate to IPv6. Participating companies include Google, FaceBook, Yahoo! and CDNs like Akamai and LimeLight Networks. My question is: where on the list are the cloud infrastructure providers? Continue reading…