Value Stream Management Meets Engineering Analytics

Value Stream Management (VSM) is about empowering delivery organizations to measure, mitigate, and monitor complexity. Simply put, it aims at improving the flow of value in your organization. The VSM Consortium recently released their highly anticipated report on The State of Value Stream Management Report 2022.  

In this article, we recap some of the findings and look at it specifically from a software engineering and DevOps point of view. Can we capture some key lessons that lead to healthier and more productive engineering teams? What has worked and what has not? Can we simplify and adapt ideas of organizational change to create a thriving engineering organization?

In the following, we present our interpretation of the report’s data and what it means for the state of engineering intelligence looking at the upcoming year.

What Is VSM and Why Does It Matter?

In laymen's terms, Value Stream Management is the science of managing the delivery of value within an organization from our (product) vision to delivering that to our customers. A key component is to map and measure how that value flows across different silos and lifecycle stages to make fact-based, data-driven decision. In turn, the impact of these decisions can be measured, creating a feedback loop of continuous learning and adaption. The result is a culture of data-driven, or rather data-assisted, management to deliver faster, more efficiently, and with more confidence.

From an engineering point of view, it is like improving our DevOps loops through measurements and continuous improvements. What we measure and how we improve are both an art and a science.

In order to obtain meaningful insight, the measurements should capture leading indicators that form the basis for metrics across silos. By tracking those value stream metrics over time, we gain understanding of our strength and weaknesses, as well as insights on how to create better engineering and delivery process. In the engineering world this is also called Engineering Intelligence.

Engineering intelligence aims to give insights to identify helpful behaviour and those observations that should triggers some alarm bells. As engineering or platform teams we naturally use metrics in the back of our head such as code review delays or build times. A typical example to look across silos are DORA metrics, which explain how frequently and reliably we can ship to our customers. DORA metrics act as a leading indicator of a smooth (or not) end-to-end flow of value from us to the customer.

When considering the flow, there are a few key metrics to keep in mind:

• Cycle/lead time: How quickly can we ship?
• Throughput: How much can we ship?
• Efficiency: What are the hiccups and bottlenecks?

And we might add:

• Quality: How reliably and at which quality can we ship?

If we have observability along each stage of the software lifecycle on those key metrics we can identify bottlenecks, process issues and flow constraints that reduce our ability to deliver. If we additionally factor in engineering allocations and resourcing, we get a picture of team and organizational health — which teams are overloaded, where should we shift resource to, and so on.

I Am an Engineering Leader. Why Should I Care About VSM?

Firstly, VSM provides a huge shift in the mindset from engineering being seen as cost center to becoming a value creation and delivery center. This makes investment into engineering a much more value supporting activity. As a result, investments in good platforms, tools, and overall DevEx are easier to argue for. Only happy engineers can deliver the most value.  

Secondly, to improve value creation we need to observe, measure, and improve on what we do and how we do things. We never get it right the first time, so data helps us to smooth out bottlenecks, measure success and puts us on a continuous improvement path.

Measuring requires capturing data — ideally ground-truth data that is up-to-date and meaningful. Engineering organizations are the engine room of many organizations. A lot of time and effort is invested in engineering, and a lot of value is being created there. As such, being able to smooth out processes, create healthy teams, and a healthy delivery pipeline brings immense benefits to organizations, and of course its developers.

The great thing is that within our engineering teams we already have all the data, we just need to capture it better and put into meaningful context. Some examples of data sources in our development lifecycle stages and our platform engineering infrastructure are planning boards and feature tracking tools, pull requests (PRs) meta-data, and our build and delivery pipelines.

While Engineering Intelligence looks into creating better software delivery organisations, VSM has a stronger emphasize on putting this into a context that is familiar with executives. Explaining engineering success in terms of VSM terminology and their metrics have become much more accepted by executives and boards and in turn helps to bring more support to creating healthy engineering groups.

The State of Value Stream Management 2022 Report

In the following, we recap some of the findings by the VSM Consortium and presents our takeaways. As mentioned, please refer to the full report for more details.

Firstly, the report is the result of survey where the respondents come from a variety of industries across the globe, and most respondents work in companies with more than 1000 employees (20% in more than 50,000 employees). This might be different from your typical high-velocity tech scale-up.  

Where Are Organizations on the VSM Journey?

The report states that most organizations have a vision of what they would like to achieve, and many are working on a plan to achieve that vision but have made limited progress on the actual execution. To agree, this is not surprising. Engineering Intelligence and VSM are relatively new disciplines and making changes across large organizations is hard. This is reflected in the survey result by noting that only a few enterprises have an active feedback and learning loop.

Who Are the Champions of VSM?

Those that drive VSM approaches in organizations are mostly C-level executives, (agile) coaches, as well as advocates in development and DevOps/platform engineering teams.  Most activities happen in agile project teams, feature delivery teams, and dedicated enabling teams. Enabling teams can, for instance, be dedicated DevEx or productivity engineering groups that are tasked to make life better for an engineering organization.

In short, the champions are those that can see the value of change. That value is seen either from an executive point of view with respect to investment decisions and revenue expectations, or from a ground-up view by those who “suffer” from obstacles or inefficiencies and are looking for hard data to justify changes.

Who Cares?

Those who care are those whose head is on the line when it comes to failed or delayed deliveries. That can be product managers or product owners, direct managers (from team leaders to VPs) and executives such as the CTO/CIO, as well as the relevant business stakeholders.

In engineering, we don’t want surprises and bottlenecks. We don’t want to be blocked and frustrated. As such, we naturally care about a smooth flow from planning to delivery.

What Is Stopping You from Getting Started With Data-Driven Approaches?

Respondents name three key reason for a more data-driven approach to value delivery: Resourcing, clear leadership, and the complexity of organizational changes. Let us park this response for a moment and look at the recommendations by the VSM community to address the above.

What Are Some of VSM Consortium’s Recommendations to Get Started?

The VSM Consortium recommends thinking big, selling a vision and defining clear goals to rally the organization behind those. Moreover, the VSM Consortium recommends establishing a connection from flow metrics to measurable business outcomes. Business outcomes here can be customer experience metrics such as product adoption, conversion rates, retention rate, P&L data etc. Basically, this recommendation is to pitch to executives and boards to get the necessary buy-in. While we see the value in this, we tend to favor a more hybrid bottom-up approach as we explain below.

Do Organizations Actually Do (Top-Down) Mapping Exercises?

The question here is, to what degree do organizations start with some (end-to-end) value mapping exercises first? The answer is not to a high degree at all. While there is a top-level desire to create better outcomes, to achieve alignment of goals, clarity, and cross-silo empathy the actual end-to-end value mapping is rarely done. Moreover, if it is done it is rarely repeated and updated.

While the desired outcome is to have data-driven insights to drive change, in reality there are many enterprise obstacles. The most cited obstacles are: The lack of resources, and the difficulty to coordinate and bring people together. This means, things don’t happen or don’t happen repeatedly, which in turn leads to a lack of (visible) change. This of course can be demotivating.

What Are the Adoption Rates for Value Stream Management Platforms?

VSM Platforms are tools and systems to capture organizational metrics and drive change. A while back Gartner was predicting 70% of companies would use some form of VSM Platform by 2023. The data from the VSM report indicates that we are not there yet. Far from it.

Organizations are particularly lacking in introspectives around the data they collect and connecting the data to business results in order to test hypotheses. Moreover, some organizations still see engineering as cost center not as a value creation unit, which in turn leads push back and higher adoption hurdles.  

Additionally, we believe that some of the more old-school tools and platforms provide less than ideal developer experiences and do not blend in well with modern tooling and cultures.

What Are the Areas of Success?

Where integrations and successes happen are, in fact, within the engineering organization of companies. Successful companies are getting insights across planning, development, build/CI, and deployments. This reflects well what we mentioned above about getting ground-truth data easily and being able to drive improvements.

The report notes that VSM data is accessed in one or several dashboards across one or more data sources using one or more tools. It seems that while today’s dashboards aggregate data they might not be very good at drilling down into causes and the raw data for identifying improvement strategies. It is one thing to measure data, another to make that data actionable enough to have a continuous improvement loop.

What Are the Most Common Value Stream Metrics?

The report cites a number of key leading indicators that teams, and organizations successfully apply. In short these include:

• Cycle time
• Throughput
• Work in progress (WIP)/load
• Flow efficiency
• Deployment frequency

We will go into more detail in a separate blog post about each of those. We will also examine what this means for different engineering personas, and which simple actions can lead to positive outcomes.

Our Take and Recommendations

Based on the findings of the State of Value Stream Management Report, the demographics of the respondents, and their organizational profiles, we will provide a few thoughts of our own. We will focus on engineering organizations, where there is a new trend of Engineering Intelligence that remedies some of the report’s findings.

Start With What You Have

We believe that pure top-down approaches create unnecessary angst and complexity that prevents teams to just get started and establish a beach head. It is the proverbial boiling the ocean approach that is hard to get buy-in for. In our view it is best to break down the problem to something manageable early on.

Engineering Intelligence and data-driven management needs to start close to where that data is generated. The ground truth. While this might require top-down organizational cover, it is much easier to get started and grow when measuring and feedback loops are quick and tight. Once the first proof points are established, it is easier to branch out and grow into more complex challenges.

The VSM report finds that many organizations are already aware of key engineering and DevOps metrics: cycle/lead time, the number of Work in Progress (WIP) items, throughput, efficiency, quality (commit size, rework/churn, review coverage, and so on). We recommend starting with what you have.

Visibility Is the Start to Making Improvements

Generally, our goals are to ship high-quality products and services faster and more reliably at higher quality. Capturing value stream metrics enables us to make processes visible, remove bottlenecks, and iterate towards continuous improvements.

Visibility gives us the ability to gain insights, to learn, and to communicate. Visibility and culture are key to connect a vision with actions as it reduces the unknowns and allows to make things happen. The first goals might be to improve engineering and team health, or plainly to achieve faster-to-market deliveries.

Once we have a data-driven beachhead established, it will be easier to drive wider organizational change and alignment. Again, visibility, communication, and culture are key.

Use Modern, Developer-Friendly Platforms

As highlighted above, you don’t want to boil the ocean before getting stared. In fact, you want some insights around your strengths and also areas for improvements on day one. Moreover, you want to encourage easy and rapid adoption of your engineering intelligence and VSM platforms. This means platforms must speak the language of its users and be easily accessible. No offence, but hardly anyone wants to see their engineering insights in SAP.

Modern Engineering Intelligence and VSM SaaS platforms are probably a great way to start — especially if you yourself are in a fast-growing organization that already uses modern and open tools, as these platforms can easily capture data through the available APIs.

These out-of-the-box connections to your existing engineering platforms and tools, a UX that is engineering leader-friendly and that speaks the right language to solve the right problems enables quick adoption. We believe that Developer Experience and Manager Experience are two items that must go hand in hand to bridge hierarchies to communicate value.

Summary and Outlook for 2023

Data-driven approaches to management and engineering are rapidly growing. Many organizations don’t see this any longer as optional, but as essential to deal with the complexities of development and product delivery, to satisfy the time-to-market pressure, and to overcome the engineering management challenges in a hybrid/remote workplace.

In essence, we see the following success factors in kickstarting a data-driven engineering culture:

1. Start where the data is. Establish a beach head.
2. Get visibility. Make small useful changes.
3. Get team buy-in. Share, communicate, iterate.

Based on out observation of pent-up market demand and grassroots approaches, we believe that 2023 will accelerate a shift to more data-driven engineering. Development teams and engineering leaders will play a critical role in making this happen. Any platform that enables this easily and you can grow with over time is worthwhile watching out for.

References

[1] VSM and Engineering Intelligence Landscape 

[2] VSM Consortium 

[3] The State of Value Stream Management Report 2022

[4] Engineering Intelligence