Moving from CI to CID

About two years ago when I started working on Cruise, I set up a Google Alert for “continuous deployment”. At first, there was a lot of noise about troop movement in Iraq. Lately, however, things have become much more relevant.

I’m happy to see this happening: It validates a lot of the theory we built into Cruise. The Cruise team and others at ThoughtWorks are as interested in deployment as they are in keeping the builds green. Getting code to production after it passed through successive stages of automated and manual testing fits with a build pipeline metaphor, so the pipeline became the core abstraction in Cruise.

The Cruise team uses Cruise as a CI and deployment server: There’s a UAT environment used by the Cruise development team, a pre-production environment used by about 100 ThoughtWorks developers, and the public release that’s sold to the world. Although the deployment was automated through Ant scripts, we left the gates between the deployment stages set to manual to make things a bit more predictable for the development team and, more importantly, our early users. A post on the XP mailing list I received at the beginning of my Cruise tenure presaged this outcome:

On November 27, 2007,  

The biggest barrier to continuous deployment
is gaining the trust that what you're going to
deploy isn't going to destabilize the environment.
This takes time and effort.

John Roth

Indeed. No developer wants the CI server to go down during the workday. We were using Cruise to build itself within a couple of iterations of starting the project, but of course the product took some time to stabilize. We rolled out our first production release two weeks after we started using Cruise in UAT. Once our user base increased, many felt that the risks of the production server being down due to a failed deployment outweighed the benefits of getting new features and bug fixes out as they became available: Users were perfectly content to wait until we’d fielded it in UAT, and even the dev team didn’t necessarily want to use every check in; it was enough to know that we could trigger a deployment on demand, as we frequently did throughout an iteration.

Should a CI server be concerned with deployment? I think so. Why do you use a CI server in the first place? Immmediate feedback. Visibility. Reporting. The peace of mind that follows. CI also implies automation, and that means executable documentation, repeatability, reduction in errors.

Don’t we also need all of these benefits in our deployment process? For many new to CI, the continuous integration process leaves off after testing, contributing to the “Last Mile” problem. I’ve seen some cases where the supposed CI build doesn’t have any tests! I hoped these were isolated incidents, but there are enough experience reports to the contrary that lead some to ask whether or not we need standards for CI. Assuming you’ve got a working CI build, why not go to the next level and deploy using the CI server?

Cruise, AnthillPro, and ControlTier have the concept of deployment baked right in. There are doubtless other servers offering these features. Either we’re all off on a tangent, or were on to something and Continuous Integration (CI) is becoming Continuous Integration & Deployment (CID). Maybe in a few years, leaving deployment out of your CI solution will seem gauche as leaving out tests.

There’s a lot more to say about how the challenge of continuous deployment affects the software lifecycle and how it fits with more established configuration management practices in CMMI and ITIL. Although I’ve since left ThoughtWorks, I’m working on a continuous deployment solution for my new employer, DRW Trading. I’ll continue to post more of my thoughts on CI and CD, and I’m looking forward to discussing more of these ideas with members of the community at CITCON North America in April.