A conversation with Gene Kim, chief technology officer, researcher and author of "The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win."
The Enterprisers Project (TEP): It has been about 18 months since the release of The Phoenix Project, your novel about DevOps. Tell us about your latest project.
Gene Kim: It's called "The DevOps Cookbook," and it was actually supposed to come out before "The Phoenix Project." The goal of the book is to put into context the cultural norms, principles, and observed patterns in high-performing organizations that enable fast flow of features from dev to ops while preserving world-class reliability and stability. For me, I think the big surprise three years into this project is that it’s as much about organizational learning as it is about key performance measures.
TEP: Can you expand on that point a little?
Gene Kim: Steven Spear from MIT expresses this very well. In essence, he says that when we study the metrics of steady performance improvement over years we see that whether it’s Toyota or a DevOps organization, improved performance is a proxy measure for organizational learning. And he really is right. An organizational learning culture is what enables some of these wildly different ways of working, whether it’s automated deployments, creating environments on demand, or proactive production — all these things that we do in DevOps. They're all created by organizations that have cultures of true organizational learning. I think that insight really drives the opening section of "The DevOps Cookbook."
TEP: Although DevOps clearly is a cultural and process shift, do you find that some in IT still think of DevOps as a service or a product that’s for sale?
Gene Kim: The optimist in me refuses to believe that anyone would fall for the idea that DevOps is something that you buy. On the other hand, there’s a striking similarity between kind of tools that large complex organizations are using for DevOps and what you would find in classic hipster unicorn DevOps companies like Netflix, Amazon, and Etsy. So maybe another learning for me is that the DevOps movement is as much a culture as it is a set of work practices, which includes automation. In other words, just having a great culture without having great work practices or technical practices is more like the Agile community, where you can have a bunch of Agile people, but without automating the right things you’re going to end up with as suboptimal outcomes.
What about waterfall culture?
TEP: If DevOps is a kind of cultural state, are there also waterfall cultures?
Gene Kim: Oh, yeah, absolutely. Conway’s Law states that the organization dictates how work is done because it produces designs that copy the organization's communication structure. So if the organization says we’re going to do things in a waterfall way, then we’re going to have a development department who hands the code off to a test department who hands it off work to the operations department. You end up with very large releases combined with low ability to find and fix defects quickly, which leads to horrendous deployment outcomes. The organizational culture leads to what gets delivered.
In DevOps organizations you end up with much smaller teams, where typically you will have dev and ops working side-by-side, and if not integrated into the same team at least working far more closely together, collaborating to not just facilitate getting the feature done, but making sure that it’s safely and continuously integrated and deployed into the production environment. You end up with a different-looking organization, as I've noted elsewhere.
TEP: In other words, you can't just say 'We're a DevOps organization' and have it be so.
Gene Kim: You can't. And a very real question related to that point is, “What does the next generation of centralized operations look like in large, complex organizations?” One model is this notion of banding operations engineers together in the same group as the feature or service team. Which is great, because now the operations people have the same goals as the service team. Another model is that operations is less in the business of doing work that comes out of the ticketing systems. Instead they’re building the automated tools that allow other people to do their work in self-service mode. The concrete example is this notion of a shared group that’s all about increasing development productivity. How do we make environments available on demand to any feature team or service team who wants to use one, and what are the deployment mechanisms they can use to get into the production environment? Basically, make the easiest route for any development team to become productive and become more reliable.
TEP: It sounds like you're saying that centralized ops is not going to disappear.
Gene Kim: No, I don’t think centralized IT operations disappears. They just provide value in a different way. People don’t submit work in tickets. Instead we’re making these automated capabilities available so that anybody can do this work without actually having to bug an operations person and wait weeks.
The role of containers in DevOps
TEP: Getting a bit more into tools and operations, there is the emergence of container technologies. What do these tools mean for DevOps?
Gene Kim: Here’s a story I love about how containers can decrease lead times and increase feedback to developers. Elisabeth Hendrickson is senior director of engineering at Pivotal Labs. When she was part of the Cloud Foundry project, their biggest problem was that whenever a developer made a code change to Cloud Foundry it would require spinning up 45 virtual machines, and it would take 30 minutes to get the Cloud Foundry instance running. Then they moved on to a project that essentially put the entire Cloud Foundry instance into containers so that you could run them on every dev laptop. After that, whenever you made a change it would take about the five minutes to get the new instance running. Quality went up, lead time went down, and the developers were finding issues faster and fixing them faster. You couldn’t do that before containers. Containers are kind of a technical miracle. Developers can not only write the unit tests and run them but also now write and run full integration tests, right on their own laptop.
TEP: With all the innovation going on in DevOps, is there a group of tools you think are worth pointing out for people to look at right now?
Gene Kim: Right now I would look at Puppet, Chef, Ansible, Salt, Hudson, Jenkins, New Relic or AppDynamics. Also Splunk, Nexus, Artifactory, statsd and Graphite. But keep your eyes open, because there are new tools and apps all the time.
Gene Kim is a multiple award winning CTO, researcher and author. He was founder and CTO of Tripwire for 13 years. He has written three books, including “The Visible Ops Handbook” and “The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win." Gene is a huge fan of IT operations, and how it can enable developers to maximize throughput of features from “code complete” to “in production,” without causing chaos and disruption to the IT environment.
