in Tools

adventures in configuration management with Puppet

I’ve started investigating higher-order system configuration management tools, in particular, Puppet, in order to help manage CBC‘s web infrastructure. About two years ago, the only player in this space was cfengine, which at the time struck me as quite functional, but also quite arcane. Puppet seems to be much easier to learn. But I’ve come up against a fundamental problem that I’m hoping more experienced Puppet users can help me with. How does one force synchronous updates to Puppet clients? In this entry, I’ll explain my use case, and hopefully get some answers as to whether Puppet is the right tool for the job.

First, some background: At CBC, we use a proprietary in-house tool known as ASC to manage application server (Apache and Tomcat) configurations across our web and Java cluster. ASC has been around for many years and I guess you could say that I’m the current maintainer; the ideas aren’t mine (I must give credit to admins of days past including Rob Fullerton, Rob Ellis and Matthew Will), but I commit the most to it. What ASC does is to retrieve an Apache or Tomcat configuration template from a revision control system, make appropriate token substitutions for the target server, "sandbox" (Apache or Tomcat instance), deploy the target file to the remote host, and restart the Apache/Tomcat instance if requested. Conceptually, this is pretty simple, and it’s worked well for years.

The ideas in ASC are sound but we are finding it increasingly difficult to maintain newer Tomcats that have a million little configuration files: server.xml, tomcat-users.xml, plus all the little context descriptors for each web application (some other day, I will describe how we manage deployment of web applications across the cluster.) ASC was only ever designed to deploy one configuration file per application. This worked well back in the Tomcat 3.x days, but is no longer as useful for Tomcat 5.5.

Thus, I have started to look at piggybacking on other tools like Puppet. The only problem is, the process I have described above is completely synchronous; that is, when an administrator commits a change to a configuration repository, he/she expects to be able to pull the trigger and deploy the change immediately. With both cfengine and Puppet it seems like changes are deployed asynchronously; the administrator commits the change and the Puppet Master picks up the change; at some point later, the client machine running puppetd contacts the Puppet Master and refreshes its catalog, and then applies the changes, restarts its own services, etc.

So my question to the configuration management community is this: Is there any configuration management tool that permits the administrator to “pull the trigger” right away and execute changes in a synchronous manner?