Struggling With Toolchain Sprawl? You're Not Alone

With more and more organizations focusing on DevOps, it’s not surprising to see the number of toolchains — and the complexity of those toolchains — multiply. Afterall, automated testing for function, security, and deliverability is at the core of improving a team’s DevOps.

But, at what point are more tools and toolchains creating more distracting work for your team than they save? Is the complexity of these automated tools actually leading to a better development lifecycle, product, or service? Of course, there’s no simple answer to what the right number of toolchains or tools is, given the complicated and diverse circumstances different teams of developers face.

Recently GitLab hired Forrester Consulting conducted a survey of 252 IT professionals across the US and Europe to learn how development teams are using their tools and toolchains. You can read their entire report here, but here are three key findings:

1. Only 22% of survey respondents reported using only one toolchain for software delivery, with 45% using three or more. Of those using three tools per toolchain or more, two-thirds were using eleven or more tools per toolchain. 59% of those with only one toolchain reported only using one to five tools. In other words, the more toolchains an organization used, the more likely those toolchains were more complex.

Integrating these complex toolchains can be just as complicated. The responding IT professionals reported that 35% of their toolchain was integrated through plug-ins and scripts, another 23% via APIs —but 20% are integrated by time-consuming and potentially insecure manual custom hard-coded solutions. Only 4% reported their tools are not integrated, while 18% relied on an out-of-the-box software delivery solution that came entirely integrated. 

30% of respondents reported struggling with integration simply because they didn’t have the required skills. We once worked with an AI company who’s engineers were brilliant, but their deployments kept crashing until AWS DevOps experts were able to automate their deployments with GitLab Runners.

2. Elaborate toolchains are making it a challenge for anyone to truly understand how their development process works. When things break, no one really knows why or how to fix it. 39% complained of a lack of visibility in their toolchains. Many teams are stuck simply completing the CI/CD process instead of innovating or working on new features. 67% responded that handing off between different teams using different tools slowed down delivery.

The resulting low visibility of complicated toolchains also presents new security concerns. Each tool will have its own requirements. 46% responded that they spend too much time and money maintaining and integrating the varied security protocols of every tool.

3. Organizations reported 42% were using an out-of-the-box solution to help manage and maintain their toolchains. For the rest, 40% relied on their development teams to maintain their toolchains, 31% on their DevOps team, 19% on a dedicated software tools team, and 10% on an operations team. Those using out-of-the-box toolchain management solutions reported improved security 47% of the time and 46% reported an increase in revenue.

Forrester also notes that choosing a toolchain management system that is capable of deploying to any cloud environment is essential, given the shift towards cloud based products and services.

Recommendations to Manage DevOps and Toolchain Sprawl?

Start using a cloud-agnostic, out-of-the-box toolchain management solution. Security issues resulting from complex toolchain was the top reported concern of organizations — and improved security was the top reported benefit of adopting them. Having a better understanding of the tools and increased visibility was another top priority for organizations looking to better track performance. Know what updates to prioritize and get the most from their toolchains.

These recommendations are supported by research in the seminal 2019 Accelerate State of DevOps Report led by Google Cloud’s Nicole Forsgren. They found that the best performing teams embraced instead of avoiding automation and integration tools, so there’s really no escaping toolchain sprawl in today’s development environment. So, what did the report recommend when it comes to tools and toolchains?

Organizations that the report categorized as the worst DevOps performers were four times as likely to rely on mostly proprietary, in-house solutions than all other higher-performing teams. Perhaps not coincidentally, Forrester reported 20% of organizations as relying on wasteful in-house, custom integration coding. This was the same percentage of low performing teams that Dr. Forsgren's found relied on in-house tools. She also found the worst DevOps performers were the most likely to rely on heavily customized versions of out-of-the-box tools.

The report also found that top performers favored tools that were simple to use. This may seem obvious, but since low visibility was one of the top reported concerns of toolchain sprawl, clearly it isn't being practiced by everyone. Choosing easy-to-use tools needs to be a focus for struggling DevOps teams to improve their performance. You can read more about the non-toolchain recommendations the 2019 State of DevOps report made here.

So how is your organization handling toolchain sprawl? Have any success stories? Nightmares? Share your experience in the comments below.

Further Reading

• Compliant DevOps.
• Building a Continuous Delivery Pipeline Using Jenkins.