How to Measure the Success of Your Analytics Projects

We recently sat down with a large pharmaceutical company to discuss their data analytics projects. What we heard wasn’t a surprise. Three of the four large analytics efforts they undertook last year had failed. The jury was still out on the fourth.

Yet, by way of contrast, here at EastBanc Technologies we are responsible for ten analytics projects at any given moment, without the mammoth-sized resources available to the large enterprise clients that we often serve.

The difference between our success rate and the failures we see has little to do with the complexity of the undertaking (although that’s a part of it) or the incredible budgets allocated; it has more to do with at what point and how frequently along the project path a success measurement is taken.

Traditionally, complex, large-scale analytics projects are so involved and unwieldy that only at the end of the project is a real success assessment made, often years after kick-off.

That’s not our approach. We weave success measurement into the entire process. That measurement and the completion of tasks become two sides of the same coin.

We do this by leveraging two approaches:

1. An Agile Methodology

Agile has been around for more than 15 years but has only recently gone mainstream. Agile is a term used in the IT world that describes a process of software or product development that stresses cross-functional collaboration over silos, adaptive planning, early delivery, and continuous improvement.

Agile breaks work into small increments and short timeframes. During this period, team members come together to report on what they did the previous day towards their iteration goal; what they’ll do today; and any impediments that may be in the way of that daily goal.

Feedback, transparency, and collaboration are key and encouraged. Team members are empowered to make suggestions about how things can be done differently and prioritize tasks more efficiently. Through constant adaptation and improvement, Agile teams can more quickly respond to changing needs, address problems as quickly as they arise, and iterate towards a higher quality data analytics product – and a lower probability of failure.

To break it down, an Agile methodology incorporates the following key features and benefits:

Weekly Sprints – With Agile, success measurement is woven into the weekly process. Weekly recaps (a.k.a. Sprint Meetings) provide a moment in time to measure progress against your original Minimal Viable Prediction (MVP) and assess if you’re on the right track. Because any failure caught at this stage is likely to be small in scale, it’s also easier to rectify. Plus, each failure is a new insight that will inform the next Sprint. For a deeper dive into the anatomy of the Sprint approach, check out: Sprint, Scrum, and Seize Success.

Business Stakeholders are Core to the Process, not on the Periphery – Another big bonus of this process is that business people are part of the process from day one. Unlike traditional waterfall approaches to software development which kept business people out of their own projects, with an Agile approach, business stakeholders are involved every single day, or at least at weekly Sprint meetings, rendering failure close to negligible.

Micro Goals Aid Success Measurement – A core facet of the Agile methodology is breaking goals down into micro goals. These are measured and evaluated on a weekly basis by all stakeholders, measuring both technical and business success.

If this process is structured right, the mere completion of a task equals a micro-success measurement. That the task was collaboratively added to the backlog in the first place means that its transparent completion by the team member, who took ownership of that task, added another mini-checkmark to the success of the overall project.

2. New Lightweight Cloud-Based, Pay-As-You-Go Technology

Alongside Agile, our technology approach also flips tradition on its head.

Legacy analytics technology forced teams to think sequentially rather than in parallel. New cloud-based business intelligence (BI) technologies are paving the way for immediate measurements and real-time, not historical, insights, allowing businesses to adjust goals along the way. Unlike monolithic systems where integration was a huge challenge, these modular, service-based architectures are relatively easy to integrate. You can also fire them up as needed and pay-as-you-go or switch providers whenever you want.

Another benefit to cloud BI is that you can start small. In the past, any BI initiative involved a huge investment, terabytes of data, and technology overhead that prohibited smaller, nimbler projects.

As data analytics becomes democratized and failure rates decrease, the role of analytics will become increasingly important. Powered by the cloud, BI technology has become an orchestrated service eliminating the need for unnecessary redundant features while minimizing vendor lock-in. Plus, executives get real-time results faster (and at a lower price point) and are energized to want more.

New Approaches Require New Mindsets

Of course, a new way of doing things requires a cultural mind shift. Adjusting to an Agile product development methodology and thinking in terms of rapid iterations involves acting like a pivoting start-up: agile and ready to change directions quickly. Yet decades-old traditions and hierarchies stand in the way.

The charge to change must be led by data and facts, not gut instincts and established routines. A pay-as-you-go approach also means that you won’t negotiate a software license anymore or have hard-set deliverables. This makes budgeting a challenge. How do you know how many licenses you’ll need the next year? The business model of the big legacy players was to provide you with the number of licenses you’ll need from day one. With the pay-as-you-go cloud BI tools, you may want to start with one and add more as you see success – pay-as-you-go cloud models give you the flexibility to do that. But people who think the old way will find it hard to adjust.

All Too Radical? Start with a Hybrid

If this path to data analytics success is more than you can bite off right now, don’t worry. Start with a mix of the traditional and MVP approaches. Instead of an MVP, choose a bigger goal, perhaps something you can revise on a quarterly basis. Even if you lose three months because your team was on the wrong track it is still better than no iteration at all and realizing only at project completion that you failed.

If you can’t include business stakeholders on a weekly basis, loop them in each quarter.

You don’t have to adopt a pure Agile approach from the get-go. Go Agile iteratively and slowly increase adoption as stakeholders see the benefits for themselves.

Take a Leap of Faith

Look at the disruptors in your industry and take a small leap of faith. It’s a risk, but taking iterative, incremental success measurements will ensure a result that truly provides value.

In the words of Amazon-founder and owner of our customer The Washington Post (which has famously adopted Agile): “There is no map, and charting a path ahead will not be easy. We will need to invent, which means we will need to experiment.”