A General Software Maturity Model

We use the word “maturity” frequently in the software business, but is there any clear definition shared by the whole industry? It is often used to refer to software or a software organization, but a project's maturity level is ultimately a subjective judgement. However, once you understand the context of the software and your organization, there are some verifiable metrics such as project activity and age that can be used to give you an idea of whether or not a piece of technology is mature enough for your needs. So what are some generally applicable maturity metrics? Given the importance of building applications on reliable, mature technology, this is an important question that we should explore.

Why We Want Maturity

In open source projects, the main benefit is having a large amount of developers using, testing, improving, and hardening the code so that it can perform reliably in most situations. There’s also a greater amount of information, documentation, and support surrounding the project when more people are working together on it and using it heavily. When a project is considered mature, another benefit that users expect is stability.

When we make significant technology decisions, we don’t want our version of that technology to become obsolete or require significant updates in a year or two. We also don’t want to have to re-learn how to use the technology because of major changes to the API. Developers always need to keep learning, but if they are forced to update or re-learn a framework every few months, it’s hard to focus on building new features or doing anything else.

Stability also refers back to the large developer community that a popular project may have. If a significant portion of the community decides to stop working on a project, the quality of the project can start to deteriorate. We want the community support behind a project to be stable. If a large swath of developers are drawn away from the project you’re using by another, newer, more exciting project, or they have a conflict with the project leadership that causes a mass exodus, then you lose a lot of the community support that makes the project useful.

How We Prove Maturity

So how do you ensure that any new technologies you might want to implement in your software are mature? Often, developers tend to think they have a pretty clear idea of which projects are mature based on the blogs and tech news they read, which is a good thing to do, but it still relies on intuition and anecdotal evidence.

What hard data can you use to support your hypothesis of whether or not a project is mature? And how do you frame that data in context? Maturity is defined very differently for Java enterprise developers versus universal JavaScript developers. This is just one example that shows why you need to establish your context first.

Context For Maturity

The standard for judging a project’s maturity depends on the technology ecosystem you’re coding in and the level of risk-aversion in your industry and organization. An agile startup or an application with microservices can switch technology stacks without too much impact on the user. A health or financial software department, however, has regulations to follow. They might also have some very old applications that can’t be migrated to a new technology without a significant user impact.

Maturity judgements can be very different between different technology ecosystems as well. While the fast-evolving universal JavaScript development community would consider Node.js and AngularJS 1.x very mature, a Java or C# organization might consider the entire Node.js ecosystem immature when compared with Java, C#, PHP, Ruby, or Python ecosystems. Who’s right? Both sides are, because maturity is relative.

In the Java ecosystem, the most mature frameworks and libraries will have been around for six to ten years (Apache Maven is 11 years old and JSF is around 15 years old, for example).  For the Node.js ecosystem, a framework created in 2009, like AngularJS, is considered very mature if it still has a strong userbase (Ember.js, a comparable framework, is four years old and Backbone.js is five years old). The most mature databases will be over ten years old (MySQL is 20 years old), while the less-mature (but still mature enough for many organizations) databases will have been around for five to six years (MongoDB is six years old).

If you’re judging the maturity of the language ecosystems themselves, the things to measure would include the number of tools that have had long lifespans and pervasiveness of the language in applications throughout the software industry. Anecdotally, the experts I’ve talked to over the years would put Java and C# in the most mature tier. PHP would be in the next highest tier with Ruby catching up, and Python after that (even though it’s been around longer than all of them). The Node.js/universal JavaScript community has an incredible amount of community contributions and is probably the most vibrant ecosystem right now. But that also means there’s a high rate of churn among the favorite frameworks and libraries in the community. In 1-2 years, the leaders in the Node.js and frontend JavaScript space could be using a completely different stack from the ones being used now.

Maturity Metrics

The context is important, but you probably noticed that it’s still heavily based on impressions and the anecdotes of community leaders. There’s not a lot of research specifically aimed at answering questions about project and ecosystem maturity. However, there are several initial common-sense metrics that bloggers in the developer space have come up with to help guide technology decisions based on maturity. Some concrete definitions of maturity focus on the rate at which bug fixes, patches, and features are released. If these updates are committed to a project on a steady schedule, and the those changes/fixes in each release rarely impact backward compatibility or usability, then a project is generally considered mature. But again, this is a flexible definition that should depend on your organization’s specific needs. That’s why you should think of creative ways to measure and cross-tabulate the following maturity metrics:

• Number of users

• Number of contributors

• Frequency of bug fixes/patches

• Number of outstanding issues

• Repository stats: GitHub stars, watching, forks, and number of other repositories using the project; or equivalent stats.

• Number of Stack Overflow questions: Tag popularity, questions answered.

• Frequency of new features

• Release label: Is it still in beta? At version 1.0? “production ready”?

• Frequency of API changes: Is the API stable and not receiving usability changes often?

• Existence of a “core” team of developers: Focused team that responds to bug reports faster, tests more rigorously, and is more strict about maintaining project guidelines.

• Amount of documentation: API, books, tutorials, and blogs. Doesn’t merely focus on the syntax but has examples for methods and properties.

• Frequency of community responses: Experienced users willing and able to help other users. Are they building extensions or working on the core framework?

• Existence of a commercial backer/steward company or non-profit: If the steward or owner company is too controlling with the project, it can cause the community to suddenly abandon support for the project.

• Number of high profile site/app examples: If the technology is used by several companies with challenging use cases, then the project is more likely to be reliable.

• Number of “horror stories”: The more of them that the community knows about, the better. Every project has them, and if they don't, it means that the project probably hasn't been tested as heavily in as many challenging situations.

Remember, it’s important to isolate these stats within the context of your technology ecosystem. A project may not seem like it has a lot of users until it’s compared with other similar tools within the ecosystem. Qualitative analysis will also be important in judging a project’s maturity too. If the larger part of an ecosystem’s developer community believes a project is mature, it should be taken into account.

Other sources for maturity metrics besides GitHub and Stack Overflow include the following sites, which offer real web technology usage stats:

• BuiltWith

• W3techs

• Wappalyzer (Includes specific high-profile users of each project)

As for what you can do with these stats, an old, but still relevant blog post by Rob Ashton has several interesting data comparisons that he did on JavaScript frameworks which you could copy or modify.

The Dangers of Clinging to Maturity

Maturity at all costs isn’t always the right thing to do. We all know that technology moves fast for a reason. Better tools and frameworks emerge over time, and if you don’t want to get left behind by competitors, you often have to consider whether the cost of migrating to a newer technology is less than the cost of potentially loosing business. Depending on the technology ecosystem and industry you’re working in, you’ll have to determine an estimated number of years you expect to stay on the same technology stack before you have to rewrite components and change large sections of that stack.

It’s especially challenging to keep up when patterns and modes of software design are constantly changing. There may be a suite of tools that is very mature, but perhaps the tools were designed in a time when applications were built differently than modern applications. It’s important to know when the technologies you are using have reached or are approaching obsolescence.

But don’t forget that as new developers enter the workforce, the industry as a whole tends to forget some of the important concepts and problems we’ve addressed in the past, and in some cases, we reinvent the wheel or do things in a less optimal way with the new frameworks we create. Developers, like technology, also have to evolve in such a way that their knowledge matures but doesn’t get left behind.