When DevOps is done correctly, it’s a sight to behold. People are working together, tools are being applied at the right times in the right places, apps are moving through the pipeline, and all the stakeholders end up happy with the outcome.
But anybody who’s built a DevOps practice knows that things don’t always go according to plan. Everybody in the process usually has the best of intentions, but sometimes teams can run into problems executing on their DevOps strategies. They encounter the dark side of DevOps – where nothing goes right, everybody’s blaming each other, and nobody has a solution.
[ Some common DevOps wisdom falls flat. Read 7 pieces of contrarian DevOps advice. ]
How do these “dark side” situations crop up? And what can you do to get back in the light? Here are a few scenarios to consider.
1. Us against them
There’s been a lot written in recent years about the benefits of breaking down silos. Companies in virtually every industry have tried to reorganize departments to get people talking to each other more, sharing more information, streamlining the decision-making processes, and aligning toward common goals.
This kind of alignment is especially important in DevOps. But it’s not always easy to achieve.
Traditionally, the Dev side and the Ops side have been at odds because they approach their jobs with different styles. Developers are creative types who want to push the limits on code and plow through any problems they encounter. The operations people are more buttoned-up and resistant to change.
Then there’s another issue: Dev and Ops pros don’t only have to get along with each other. They must work in concert with the business side of the organization as well. Getting all sides to work together – ridding the organization of the “us against them” mentality – is critical to DevOps success.
[ Read our related article: 3 DevOps roadblocks and how to beat them. ]
Organizational issues create tension, resulting in difficult, friction-laden handoffs between teams. The problems often show up in the software delivery process and in the software itself in the form of quality gaps, inconsistent user experiences, and other issues.
Organizations need to acknowledge that culture will have a direct bearing on DevOps success, and commit to taking on those challenges. Organizations need to align, communicate, and be inclusive.
2. Poorly defined KPIs
The way a team defines success ultimately determines whether the team is successful. A team can appear to do a great job based on the KPIs it created. But if the KPIs aren’t appropriate for the specific task, the initiative may fall several steps short of actual success. As with any project, effectively communicating the metrics of a project is critical to success. It’s equally important to think through the KPIs being set up for each project and not just recycle the same ones every time.
Going back to working across departments and breaking down barriers, the KPIs that the developers are using should also tie to a business KPI, such as the number of new customers acquired or the percent of revenue increased. This will show velocity and help the entire business work more effectively, rather than just running at a speed that’s not manageable across departments.
For example, if a quality engineering team wants to measure speed, it might create a metric based on the number of tests executed per sprint. Humans are driven by incentives, and the natural tendency of the team could be to add more and more tests without archiving the tests that are outdated. What if the team tried to operate with fewer tests? Fewer tests would lower the test cycle time. So instead of sheer quantity, a team should focus on the coverage and effectiveness of tests it’s doing.
[ Read also: DevOps metrics: Are you measuring what matters? ]
3. Deploying code out of step
Sometimes teams get ahead of themselves and run applications that haven’t been properly tested. Teams should treat their CD pipelines as applications themselves by checking for and eliminating bugs and errors in CD pipeline code. When teams don’t treat CD pipelines as applications, there is a significant cost and/or risk involved.
[ Need a primer on continuous integration/continuous delivery? Read What is CI/CD? ]
Teams sometimes run in parallel during CI/CD, which can speed up their automated testing and shorten feedback cycles. To avoid this, make sure deployment is not executing as a parallel step. That misconfiguration will cause your code to deploy at the same time the tests are running, regardless of their exit codes, pretty much defeating the purpose of automated testing before a deploy.
To avoid these issues, teams should build, manage, and create staging environments. First, build from dev to test to deployment starting from the left-hand side of the process. While teams may not be able to define an ideal automation solution completely on the first pass, going left to right encourages teams to focus on what the life cycle of an artifact looks like.
Second, teams should define their CD pipeline using an everything-as-code or pipeline-as-code approach. And finally, establish staging environments or test pipelines to validate the CD process.
Following these steps will give your teams an opportunity to iterate changes against application code, do some initial scans, and collaborate to define your CD pipeline.
Working together, setting the right KPIs, and coding in the correct order aren’t the be-all-end-all when it comes to avoiding DevOps problems. But these steps can help. Keeping these things in mind can help bring some light back into your organizations’ DevOps practices.
[ Read also: DevOps for doubters: How to deal with 9 kinds of people who push back.]
