8 Metrics for Rapidly Scaling Dev Teams

Between November 2008 and December 2014, I transitioned from senior software engineer to dev team lead to Director of Engineering to VP of Engineering at CloudLock. During that time we grew from one office with 5 devs to four offices with 75 devs. 

Let’s just say it was a crazy 6 years :-) 

I’ve written a lot about the transformation I went through as a leader - most recently Promoted from Dev to Team Lead: 8 Things They Didn’t Tell Me. 

I also often get asked about the technical side of scaling a team. 

Scaling a software organization is the ultimate challenge. Sometimes you’re feeling unstoppable, and sometimes it can feel like you’re Little Mac vs. Tyson in NES Punch-Out. A wildly diverse set of skills is being tested, here, including: 

If you are in the team-scaling phase, think of it as a blessing. Your abilities and experience will level up 100%, guaranteed. The thing is, you likely need to make changes to your organization in order to gain the expected ROI from team growth. Here are some things to keep in mind as your team grows.

Technical Readiness

Team growth changes everything. Bottlenecks in your value delivery pipeline become exposed, quality issues become exacerbated (think about customer growth), and things that you were able to get away with when your team was small no longer fly. No worries, though! If you can be mindful about your organization’s test methodology, delivery independence, and codebase health, then you’ll have a solid starting point for success.

Test Methodology

Test methodology impacts quality, delivery throughput, team setup, and, ultimately, an organization’s productivity. As your team scales, the importance of completing fast test cycles increases, and, as you might expect, so will the penalties for inefficient test practices. If you have fast feedback and a high-quality test cycle that is fully automated, congrats! Keep that going. If your test methodology is not up to par, red alert! Improve this now. 

Side note: Test methodology is fascinating. I believe manual “testing” can still be useful for customer value confirmation, but your automated testing has to cover everything else. Who performs the manual “testing” is also up for debate. Personally, I think it should be the product owner. Ryan Schaffer, franchise owner of Code Ninjas, has extremely insightful thoughts about this based on his QA leadership experiences. You should ask him more about it!

My advice to software leaders regarding testability is to measure, understand, and continuously be conscious of the following metrics.

Metric #1: Test Cycle Time: Time from code completion to quality feedback loop completion.

This is a throughput metric that brings to light a crucial piece of your value delivery pipeline. You will see significant productivity gains by improving your test cycle time. Here is another way to think about it. The faster you provide your people with quality feedback, the faster they can iterate to improve the product.

Metric #2: Issue Rate: Bug rate in production, pre-production and support.

This is a quality metric, and it compliments Test Cycle Time, nicely. You want a fast feedback loop but you cannot sacrifice quality. You need to be aware of how often bugs are being found and where in the pipeline they are being discovered. Productivity will increase as bugs are found “further left”.

Side Note: I almost Forgot About Dre. Check out Defect Removal Efficiency (DRE).

Metric #3: Automated Test Coverage: Degree of the code executed when a test suite runs.

This is an activity metric, and I advise this measurement, cautiously. Like all metrics, it must be used responsibly. Test coverage is important, but only with complimentary metrics like issue rate and cycle time. It is important to understand how much effort your team is putting into automated testing and the trend of your test coverage.

There are other useful metrics in the testability category. However, I believe measuring and visualizing these three metrics provides a solid baseline that promotes the mindset needed to increase productivity as your team grows.

Metric #4: Team Delivery Independence

When more people join an organization there are, inevitably, more software teams that need to be created. As you create more teams, the importance of delivery independence increases. This means that each team needs to have the ability to get value into production at their desired rate with zero to minimal dependence on other teams. Without this, you will struggle to increase your team size and increase your productivity. There are two areas that are important to highlight in this category.

Team Performance Metrics

Visibility into what is happening within each team is crucial. I have developed a list of 17 metrics I use to measure team performance. Key performance areas need to be measured and visualized on a per-team basis because each team will have different observable challenges. Based on my experience, I believe delivery independence is the most important area of focus for technical readiness. More specifically, I recommend paying close attention to the time from code merge to production release. It’s one part of your overall Cycle Time that is a great signal for delivery independence and efficiency. This metric needs to stay the same or improve as your team grows. 

Want to see your team's Cycle Time and other performance metrics?
Click here to get started with our free 21 days free trial. In less than 5 minutes you’ll see trends from your past 5 iterations. 

Architecture and Design

The way in which your system is architected will impact the delivery independence of your teams. This is one reason why services are a good idea. Make sure that each team has the ability to own the lifecycle of their services. Your architecture and design will either hinder or promote this ability which, in turn, influences productivity.

It is also important to make sure that each of your teams have all of the necessary skill sets needed to deliver value. For now, we’ll save a deeper dive into team formation and delivery independence for a future post.

Codebase Health

The state of your codebase requires constant mindfulness. If you are about to bring a bunch of new hires into your organization, you want them to hit the ground running. The faster they can do this, the better.

Metric #5: Commenting ratio: The proportion of commenting text to code.

The quality of your codebase will impact the journey to productivity for all new hires. Part of having a quality codebase is the ability for new contributors to understand what is happening quickly. Proper commenting helps a lot.

Side Note: I’ve heard the theory around, “great code should explain itself”. I’m not buying it, and most of the leaders I meet with agree. That line tends to evoke an eye-roll.

Metric #6: Documentation ratio: The proportion of documentation text to code.

This is another quality metric and it is similar to commenting ratio. Having the proper documentation in place will increase context for new hires and decrease time to productivity.

Metric # 7: Maintainability score: Think of this as similar to Halstead’s complexity measures.

These metrics provide an insight into the complexity of the codebase and can translate into time to productivity for new contributors.

Metric #8: Standardization error rate: Amount of standardization and quality errors (things like a function ending early) per lines of code.

This one is interesting. Each organization has different practices when it comes to code standards ranging from nothing to strict. I think it’s important to decide where you want to fall on the spectrum and communicate your standards prior to team growth. Your new contributors are likely to carry forth whatever you have in place upon joining.

Technical readiness is an essential part of scaling a software organization. If you are able to measure, visualize, and be aware of trends in each of the categories above, you will have a great shot at moving your organization in the right direction.

At CloudLock, it was really hard for me to get all of this data frequently enough to use it in our iteration retros and planning meetings. That’s why I started LinearB. We correlate Git and Jira data to show dev teams and leaders the most important facts to help you

Proactively identify blocked teammates
Run data-driven stand-ups and retros
Remove dev cycle process friction
Scale your team more smoothly
Predict delays and ship faster

You can get started for free.

Let me know in the comments if I missed any metrics you find helpful for start-ups or scaling teams. I would love to hear from you!